Jul 10 18:25:54.683603 ip-10-0-138-224 systemd[1]: Starting Kubernetes Kubelet... Jul 10 18:25:55.048032 ip-10-0-138-224 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:25:55.048032 ip-10-0-138-224 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 18:25:55.048032 ip-10-0-138-224 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:25:55.048032 ip-10-0-138-224 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 18:25:55.048032 ip-10-0-138-224 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:25:55.049910 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.049813 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 18:25:55.054450 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054428 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:25:55.054450 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054450 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054454 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054458 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054461 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054464 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054467 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054470 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054472 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054475 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054478 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054481 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054484 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054487 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054489 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054492 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054495 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054498 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054500 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054503 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054506 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054509 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:25:55.054529 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054512 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054515 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054518 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054527 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054532 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054537 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054540 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054544 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054547 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054550 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054553 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054556 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054559 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054562 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054565 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054567 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054570 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054573 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054577 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054581 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:25:55.055030 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054584 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054587 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054590 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054593 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054597 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054600 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054603 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054606 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054609 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054612 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054615 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054618 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054622 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054626 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054629 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054632 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054635 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054638 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054641 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:25:55.055557 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054644 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054647 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054649 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054652 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054654 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054658 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054661 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054665 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054668 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054670 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054673 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054676 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054678 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054681 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054684 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054687 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054691 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054693 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054696 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:25:55.056099 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054699 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054702 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054705 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054708 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054710 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054713 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054716 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054719 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054722 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054725 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054728 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054731 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054735 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054737 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054740 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054743 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054746 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054749 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.054751 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055558 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:25:55.056586 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055566 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055569 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055573 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055576 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055579 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055582 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055585 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055588 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055598 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055601 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055604 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055607 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055610 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055612 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055615 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055618 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055621 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055624 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055626 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055630 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:25:55.057159 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055634 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055637 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055640 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055643 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055645 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055648 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055651 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055654 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055656 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055659 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055662 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055665 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055668 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055670 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055673 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055676 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055679 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055681 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055684 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055687 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:25:55.057702 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055689 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055696 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055699 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055701 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055705 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055707 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055710 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055713 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055715 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055720 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055724 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055728 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055732 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055735 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055738 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055741 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055744 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055747 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055750 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:25:55.058245 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055752 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055755 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055758 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055760 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055763 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055766 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055768 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055771 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055774 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055776 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055779 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055782 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055786 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055788 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055791 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055794 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055797 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055801 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055812 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:25:55.058765 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055816 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055821 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055826 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055830 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055833 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055836 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055839 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055842 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055845 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055848 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055851 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055855 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055858 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055861 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055865 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055867 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055870 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055873 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055876 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:25:55.059255 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.055879 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056328 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056335 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056338 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056341 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056344 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056346 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056349 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056352 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056356 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056359 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056362 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056365 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056368 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056370 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056373 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056376 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056379 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056382 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:25:55.059751 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056385 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056387 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056390 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056393 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056396 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056399 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056402 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056406 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056409 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056412 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056415 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056417 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056422 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056426 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056429 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056432 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056435 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056438 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056441 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:25:55.060240 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056445 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056448 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056452 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056455 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056459 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056462 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056465 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056468 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056471 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056474 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056477 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056480 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056483 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056486 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056489 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056492 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056494 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056497 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056500 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056505 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:25:55.060749 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056508 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056511 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056514 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056517 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056520 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056523 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056525 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056528 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056531 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056533 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056536 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056539 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056542 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056545 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056548 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056551 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056555 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056559 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056565 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056568 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:25:55.061338 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056572 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056575 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056578 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056581 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056584 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056586 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056589 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056592 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056595 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056598 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056601 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056605 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056608 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056610 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056613 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056616 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056619 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056622 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056624 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056627 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:25:55.061873 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056630 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.056633 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056715 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056723 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056731 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056736 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056741 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056744 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056749 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056754 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056760 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056764 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056768 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056771 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056774 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056778 2565 flags.go:64] FLAG: --cgroup-root="" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056781 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056785 2565 flags.go:64] FLAG: --client-ca-file="" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056788 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056791 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056795 2565 flags.go:64] FLAG: --cluster-domain="" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056798 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056802 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056806 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 18:25:55.062407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056810 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056814 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056817 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056821 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056826 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056831 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056836 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056841 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056846 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056851 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056857 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056860 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056863 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056867 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056872 2565 flags.go:64] FLAG: --enable-server="true" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056875 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056881 2565 flags.go:64] FLAG: --event-burst="100" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056885 2565 flags.go:64] FLAG: --event-qps="50" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056890 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056894 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056897 2565 flags.go:64] FLAG: --eviction-hard="" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056901 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056904 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056907 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056910 2565 flags.go:64] FLAG: --eviction-soft="" Jul 10 18:25:55.063023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056914 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056917 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056920 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056923 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056927 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056930 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056933 2565 flags.go:64] FLAG: --feature-gates="" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056938 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056941 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056944 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056948 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056951 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056955 2565 flags.go:64] FLAG: --help="false" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056958 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056961 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056965 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056968 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056971 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056975 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056978 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056981 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056984 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056988 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 18:25:55.063749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056991 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.056995 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057000 2565 flags.go:64] FLAG: --kube-reserved="" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057003 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057007 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057011 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057014 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057017 2565 flags.go:64] FLAG: --lock-file="" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057020 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057023 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057027 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057033 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057048 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057051 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057055 2565 flags.go:64] FLAG: --logging-format="text" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057058 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057062 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057066 2565 flags.go:64] FLAG: --manifest-url="" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057069 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057074 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057077 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057082 2565 flags.go:64] FLAG: --max-pods="110" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057085 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057088 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057091 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 18:25:55.064384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057095 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057098 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057101 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057105 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057114 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057117 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057120 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057124 2565 flags.go:64] FLAG: --pod-cidr="" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057127 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057133 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057137 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057141 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057145 2565 flags.go:64] FLAG: --port="10250" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057148 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057151 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0702c5052ace3336a" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057155 2565 flags.go:64] FLAG: --qos-reserved="" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057158 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057161 2565 flags.go:64] FLAG: --register-node="true" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057165 2565 flags.go:64] FLAG: --register-with-taints="" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057169 2565 flags.go:64] FLAG: --registry-burst="10" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057172 2565 flags.go:64] FLAG: --registry-qps="5" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057175 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057180 2565 flags.go:64] FLAG: --reserved-memory="" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057184 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057187 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 18:25:55.065052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057190 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057193 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057196 2565 flags.go:64] FLAG: --runonce="false" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057200 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057203 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057206 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057209 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057212 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057216 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057220 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057226 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057231 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057236 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057240 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057243 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057246 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057250 2565 flags.go:64] FLAG: --system-cgroups="" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057253 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057259 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057262 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057280 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057285 2565 flags.go:64] FLAG: --tls-min-version="" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057288 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057291 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057294 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 18:25:55.065750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057297 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057300 2565 flags.go:64] FLAG: --v="2" Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057305 2565 flags.go:64] FLAG: --version="false" Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057309 2565 flags.go:64] FLAG: --vmodule="" Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057315 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057318 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057473 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057477 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057480 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057483 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057486 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057489 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057492 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057494 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057497 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057500 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057503 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057506 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057509 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057512 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057514 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:25:55.066437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057518 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057520 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057523 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057526 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057529 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057532 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057535 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057538 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057541 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057544 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057547 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057550 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057552 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057555 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057558 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057563 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057566 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057569 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057571 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057574 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:25:55.066981 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057577 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057581 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057585 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057588 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057590 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057593 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057596 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057598 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057601 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057604 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057606 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057609 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057611 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057614 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057617 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057620 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057623 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057626 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057629 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:25:55.067572 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057631 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057635 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057638 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057640 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057643 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057645 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057648 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057651 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057655 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057657 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057660 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057663 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057665 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057668 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057671 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057674 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057677 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057680 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057683 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:25:55.068085 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057686 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057688 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057691 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057694 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057698 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057703 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057706 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057710 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057713 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057717 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057720 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057723 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057726 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057729 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057732 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057735 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057738 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057740 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057743 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:25:55.068704 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057745 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057748 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057752 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057755 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057758 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057760 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.057763 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.057769 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.065586 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.065608 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.066345 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.066354 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066416 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066422 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:25:55.069206 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066427 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066430 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066433 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066436 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066439 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066443 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066446 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066449 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066452 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066455 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066458 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066460 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066463 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066466 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066469 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066472 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066474 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066477 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066479 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066482 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:25:55.069599 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066485 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066488 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066491 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066493 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066496 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066499 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066502 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066506 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066509 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066512 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066515 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066519 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066523 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066526 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066529 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066533 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066538 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066541 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066544 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:25:55.070134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066547 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066552 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066556 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066559 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066562 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066566 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066602 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066606 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066609 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066612 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066615 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066618 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066621 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066624 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066626 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066629 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066632 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066634 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066637 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:25:55.070647 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066640 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066643 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066648 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066651 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066654 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066656 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066659 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066662 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066665 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066667 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066670 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066673 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066676 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066679 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066681 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066684 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066687 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066689 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066693 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066695 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:25:55.071143 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066698 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066701 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066704 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066706 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066709 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066712 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066715 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066718 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066721 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066723 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066726 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066729 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066732 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066735 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066738 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066741 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066744 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066747 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:25:55.071697 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066750 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.066755 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066901 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066908 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066911 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066914 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066917 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066920 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066924 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066930 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066935 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066940 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066944 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066950 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066953 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:25:55.072171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066956 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066960 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066964 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066969 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066972 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066975 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066978 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066981 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066985 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066988 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066991 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066994 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.066998 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067001 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067004 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067008 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067011 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067014 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067017 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067020 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:25:55.072616 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067023 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067026 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067028 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067031 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067034 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067037 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067039 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067042 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067045 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067048 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067050 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067054 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067056 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067059 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067062 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067065 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067067 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067070 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067073 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067075 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:25:55.073134 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067078 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067081 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067084 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067086 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067089 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067093 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067095 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067098 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067101 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067104 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067107 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067110 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067112 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067115 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067118 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067120 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067123 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067126 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067128 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067131 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:25:55.073889 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067134 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067136 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067140 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067144 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067147 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067149 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067152 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067155 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067157 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067160 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067163 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067165 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067168 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067172 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067175 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067178 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067181 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067185 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067188 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:25:55.074738 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067191 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067193 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067197 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067200 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067202 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067205 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:55.067208 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.067215 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.067404 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.069754 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.071176 2565 server.go:1019] "Starting client certificate rotation" Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.071307 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:25:55.075440 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.071349 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:25:55.076782 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.076761 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:25:55.078043 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.078023 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:25:55.087706 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.087687 2565 log.go:25] "Validated CRI v1 runtime API" Jul 10 18:25:55.090544 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.090415 2565 log.go:25] "Validated CRI v1 image API" Jul 10 18:25:55.091623 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.091603 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 18:25:55.094338 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.094316 2565 fs.go:135] Filesystem UUIDs: map[37ce5bf8-f0cc-4f90-8203-60e4222f004f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ddcc0f77-f484-43de-a8c8-7b43ddf55103:/dev/nvme0n1p3] Jul 10 18:25:55.094421 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.094336 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 18:25:55.101207 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.101076 2565 manager.go:216] Machine: {Timestamp:2026-07-10 18:25:55.099764199 +0000 UTC m=+0.305604269 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100216 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cd0fd3b05a27b53d01b8ce8949f9c SystemUUID:ec2cd0fd-3b05-a27b-53d0-1b8ce8949f9c BootID:2b25f681-ca04-4db7-8476-6edb78b989b5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:41:66:71:dc:a5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:41:66:71:dc:a5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:cd:cd:40:95:6e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 18:25:55.101207 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.101204 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 18:25:55.101329 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.101312 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 18:25:55.104218 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.104189 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 18:25:55.104404 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.104223 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-224.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 18:25:55.104966 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.104954 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 18:25:55.105019 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.104973 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 18:25:55.105019 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.104984 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:25:55.105648 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.105637 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:25:55.106626 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.106615 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 18:25:55.107960 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.107941 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:25:55.110891 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.110878 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:25:55.111031 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.111021 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 18:25:55.111074 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.111054 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 10 18:25:55.111074 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.111065 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 18:25:55.111165 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.111077 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 18:25:55.111165 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.111087 2565 kubelet.go:398] "Adding apiserver pod source" Jul 10 18:25:55.111165 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.111096 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 18:25:55.111621 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.111603 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:25:55.111698 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.111627 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:25:55.113840 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.113824 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 18:25:55.116465 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.116438 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 18:25:55.116566 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.116498 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 18:25:55.117382 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117355 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 18:25:55.117382 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117386 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 18:25:55.117537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117398 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 18:25:55.117537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117408 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 18:25:55.117537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117428 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 18:25:55.117537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117437 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 18:25:55.117537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117450 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 18:25:55.117537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117459 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 18:25:55.117537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117470 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 18:25:55.117537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117479 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 18:25:55.117537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117493 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 18:25:55.117537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.117502 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 18:25:55.118302 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.118281 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 18:25:55.118379 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.118311 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 18:25:55.122208 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.122188 2565 server.go:1268] "Started kubelet" Jul 10 18:25:55.122543 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.122302 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 18:25:55.122667 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.122651 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 18:25:55.122801 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.122316 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 18:25:55.123138 ip-10-0-138-224 systemd[1]: Started Kubernetes Kubelet. Jul 10 18:25:55.128324 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.128297 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 18:25:55.137124 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.137101 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-224.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 18:25:55.137564 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.137548 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 18:25:55.137645 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.137616 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 18:25:55.137704 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.137689 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-224.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 18:25:55.137876 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.137865 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 10 18:25:55.143921 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.142972 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-224.ec2.internal.18c100ef4817fc80 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-224.ec2.internal,UID:ip-10-0-138-224.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-224.ec2.internal,},FirstTimestamp:2026-07-10 18:25:55.122142336 +0000 UTC m=+0.327982385,LastTimestamp:2026-07-10 18:25:55.122142336 +0000 UTC m=+0.327982385,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-224.ec2.internal,}" Jul 10 18:25:55.144316 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.144300 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 18:25:55.144379 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.144323 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 18:25:55.144529 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.144508 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:55.145008 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.144988 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 18:25:55.145008 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.144990 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 18:25:55.145143 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.145020 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 18:25:55.145193 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.145142 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 10 18:25:55.145193 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.145153 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 10 18:25:55.145607 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.145591 2565 factory.go:55] Registering systemd factory Jul 10 18:25:55.145607 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.145609 2565 factory.go:223] Registration of the systemd container factory successfully Jul 10 18:25:55.145822 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.145808 2565 factory.go:153] Registering CRI-O factory Jul 10 18:25:55.145822 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.145824 2565 factory.go:223] Registration of the crio container factory successfully Jul 10 18:25:55.145957 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.145889 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 18:25:55.145957 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.145914 2565 factory.go:103] Registering Raw factory Jul 10 18:25:55.145957 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.145927 2565 manager.go:1196] Started watching for new ooms in manager Jul 10 18:25:55.146848 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.146705 2565 manager.go:349] Starting recovery of all containers Jul 10 18:25:55.158584 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.158425 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-224.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 18:25:55.158584 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.158451 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 18:25:55.160233 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.160074 2565 manager.go:354] Recovery completed Jul 10 18:25:55.166474 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.166455 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:25:55.167097 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.167076 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gs99h" Jul 10 18:25:55.168621 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.168601 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:25:55.168690 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.168633 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:25:55.168690 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.168649 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:25:55.169282 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.169267 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 18:25:55.169282 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.169280 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 18:25:55.169412 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.169297 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:25:55.169412 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.169313 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:25:55.170028 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.170014 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:25:55.170079 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.170040 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:25:55.170079 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.170050 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:25:55.171634 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.171618 2565 policy_none.go:49] "None policy: Start" Jul 10 18:25:55.171715 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.171640 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 18:25:55.171715 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.171655 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 18:25:55.173768 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.173755 2565 policy_none.go:47] "Start" Jul 10 18:25:55.174903 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.174887 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gs99h" Jul 10 18:25:55.181887 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.181781 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-224.ec2.internal.18c100ef4add2016 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-224.ec2.internal,UID:ip-10-0-138-224.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-224.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-224.ec2.internal,},FirstTimestamp:2026-07-10 18:25:55.16861647 +0000 UTC m=+0.374456522,LastTimestamp:2026-07-10 18:25:55.16861647 +0000 UTC m=+0.374456522,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-224.ec2.internal,}" Jul 10 18:25:55.217969 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.217949 2565 manager.go:337] "Starting Device Plugin manager" Jul 10 18:25:55.218049 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.217992 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 18:25:55.218049 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.218006 2565 server.go:87] "Starting device plugin registration server" Jul 10 18:25:55.218344 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.218329 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 18:25:55.218402 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.218345 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 18:25:55.218493 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.218468 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 18:25:55.218659 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.218589 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 18:25:55.218659 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.218614 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 18:25:55.219265 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.219241 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 18:25:55.219368 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.219319 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:55.286659 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.286614 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 18:25:55.289151 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.287976 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 18:25:55.289151 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.288015 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 18:25:55.289151 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.288045 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 18:25:55.289151 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.288098 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 18:25:55.290926 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.290900 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:25:55.319017 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.318938 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:25:55.320074 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.320052 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:25:55.320189 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.320087 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:25:55.320189 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.320100 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:25:55.320189 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.320142 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.330252 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.330228 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.330348 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.330256 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-224.ec2.internal\": node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:55.358072 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.358042 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:55.388303 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.388255 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-224.ec2.internal"] Jul 10 18:25:55.388471 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.388342 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:25:55.390871 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.390856 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:25:55.390942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.390887 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:25:55.390942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.390897 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:25:55.393472 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.393459 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:25:55.393591 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.393574 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.393649 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.393625 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:25:55.394293 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.394268 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:25:55.394293 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.394283 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:25:55.394473 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.394300 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:25:55.394473 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.394309 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:25:55.394473 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.394316 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:25:55.394473 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.394323 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:25:55.396859 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.396836 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.396964 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.396870 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:25:55.397821 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.397803 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:25:55.397911 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.397836 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:25:55.397911 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.397848 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:25:55.413040 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.413011 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-224.ec2.internal\" not found" node="ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.417556 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.417538 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-224.ec2.internal\" not found" node="ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.458657 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.458629 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:55.547483 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.547450 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8c74eff82b7e1b9da47b5565ae9756ea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal\" (UID: \"8c74eff82b7e1b9da47b5565ae9756ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.547592 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.547493 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c74eff82b7e1b9da47b5565ae9756ea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal\" (UID: \"8c74eff82b7e1b9da47b5565ae9756ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.547592 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.547525 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9979441dfb575ea6a2636357cc791013-config\") pod \"kube-apiserver-proxy-ip-10-0-138-224.ec2.internal\" (UID: \"9979441dfb575ea6a2636357cc791013\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.558953 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.558920 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:55.648279 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.648191 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8c74eff82b7e1b9da47b5565ae9756ea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal\" (UID: \"8c74eff82b7e1b9da47b5565ae9756ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.648345 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.648275 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c74eff82b7e1b9da47b5565ae9756ea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal\" (UID: \"8c74eff82b7e1b9da47b5565ae9756ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.648345 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.648310 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9979441dfb575ea6a2636357cc791013-config\") pod \"kube-apiserver-proxy-ip-10-0-138-224.ec2.internal\" (UID: \"9979441dfb575ea6a2636357cc791013\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.648345 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.648328 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8c74eff82b7e1b9da47b5565ae9756ea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal\" (UID: \"8c74eff82b7e1b9da47b5565ae9756ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.648442 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.648351 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9979441dfb575ea6a2636357cc791013-config\") pod \"kube-apiserver-proxy-ip-10-0-138-224.ec2.internal\" (UID: \"9979441dfb575ea6a2636357cc791013\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.648442 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.648403 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c74eff82b7e1b9da47b5565ae9756ea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal\" (UID: \"8c74eff82b7e1b9da47b5565ae9756ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.659118 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.659084 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:55.718462 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.718437 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.720757 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:55.720739 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-224.ec2.internal" Jul 10 18:25:55.759649 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.759604 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:55.860559 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.860518 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:55.961672 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:55.961584 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:56.061939 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:56.061898 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:56.070695 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.070669 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 18:25:56.070883 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.070861 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 18:25:56.162605 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:56.162575 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:56.181350 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.181314 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 18:20:55 +0000 UTC" deadline="2028-04-19 21:24:50.940062408 +0000 UTC" Jul 10 18:25:56.181350 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.181344 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15578h58m54.758721895s" Jul 10 18:25:56.223244 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.223169 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:25:56.263647 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:56.263610 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-224.ec2.internal\" not found" Jul 10 18:25:56.278670 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.278642 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:25:56.279910 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.279889 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:25:56.280004 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.279928 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:25:56.280004 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.279941 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:25:56.298407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.298383 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:25:56.344847 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.344798 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" Jul 10 18:25:56.358906 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.358882 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:25:56.360123 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.360106 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-224.ec2.internal" Jul 10 18:25:56.370428 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.370400 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:25:56.471261 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.471234 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:25:56.486970 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:56.486929 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9979441dfb575ea6a2636357cc791013.slice/crio-a0cfd10b2f92e8d83bbc8fc5d0b9f1e34f9d83c4624f40f124ece2055424e1c9 WatchSource:0}: Error finding container a0cfd10b2f92e8d83bbc8fc5d0b9f1e34f9d83c4624f40f124ece2055424e1c9: Status 404 returned error can't find the container with id a0cfd10b2f92e8d83bbc8fc5d0b9f1e34f9d83c4624f40f124ece2055424e1c9 Jul 10 18:25:56.487314 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:56.487292 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c74eff82b7e1b9da47b5565ae9756ea.slice/crio-292e38f8b9eace2170495a5c66dd92b0a436ef3384e52f6657e889163e501984 WatchSource:0}: Error finding container 292e38f8b9eace2170495a5c66dd92b0a436ef3384e52f6657e889163e501984: Status 404 returned error can't find the container with id 292e38f8b9eace2170495a5c66dd92b0a436ef3384e52f6657e889163e501984 Jul 10 18:25:56.492607 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:56.492586 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:25:57.111880 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.111852 2565 apiserver.go:52] "Watching apiserver" Jul 10 18:25:57.123463 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.123431 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 18:25:57.124742 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.124712 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-vdnl4","openshift-multus/network-metrics-daemon-tvbm5","openshift-network-diagnostics/network-check-target-mp2lj","kube-system/kube-apiserver-proxy-ip-10-0-138-224.ec2.internal","openshift-image-registry/node-ca-qjh75","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal","openshift-multus/multus-q8vll","openshift-network-operator/iptables-alerter-wm6v8","openshift-ovn-kubernetes/ovnkube-node-4ft2m","kube-system/global-pull-secret-syncer-8kx2z","kube-system/konnectivity-agent-f6s8j","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g","openshift-cluster-node-tuning-operator/tuned-hck9f","openshift-dns/node-resolver-7bh5c"] Jul 10 18:25:57.127521 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.127497 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.129895 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.129872 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:25:57.130003 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.129952 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:25:57.131033 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.131005 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 18:25:57.131128 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.131005 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 18:25:57.131184 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.131005 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 18:25:57.131184 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.131077 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 18:25:57.131346 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.131321 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 18:25:57.131613 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.131593 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7ltt7\"" Jul 10 18:25:57.132240 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.132162 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:25:57.132341 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.132247 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:25:57.134584 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.134564 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qjh75" Jul 10 18:25:57.137031 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.137013 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.137558 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.137539 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 18:25:57.137660 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.137606 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 18:25:57.137793 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.137777 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 18:25:57.137874 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.137838 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-57nsd\"" Jul 10 18:25:57.139383 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.139360 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wm6v8" Jul 10 18:25:57.139843 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.139822 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 18:25:57.139975 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.139916 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rlprg\"" Jul 10 18:25:57.141574 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.141555 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:25:57.141927 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.141911 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.142331 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.142213 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 18:25:57.142702 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.142680 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:25:57.142926 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.142906 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 18:25:57.143176 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.143160 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v5q9f\"" Jul 10 18:25:57.145701 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.144412 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:57.145701 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.144509 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:25:57.145701 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.144834 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 18:25:57.145701 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.145301 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 18:25:57.145701 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.145462 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 18:25:57.145701 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.145545 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 18:25:57.145701 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.145581 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 18:25:57.145701 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.145683 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 18:25:57.146038 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.145845 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g5lc8\"" Jul 10 18:25:57.146038 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.145910 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 18:25:57.146868 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.146836 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:25:57.149841 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.149725 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vcvjp\"" Jul 10 18:25:57.149841 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.149738 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 18:25:57.150002 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.149843 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.150813 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.150794 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 18:25:57.153246 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.153201 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.153929 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.153629 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lbtc8\"" Jul 10 18:25:57.153929 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.153725 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 18:25:57.153929 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.153797 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 18:25:57.153929 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.153846 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 18:25:57.155354 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.155324 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7bh5c" Jul 10 18:25:57.156994 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.156967 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-os-release\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.157066 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157015 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-log-socket\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.157066 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157055 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a8ca608-59d8-4480-925d-00d01e6eda10-ovn-node-metrics-cert\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.157138 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157090 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/098c1a69-137c-45c4-a356-ed61de961a17-dbus\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:57.157138 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157124 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-system-cni-dir\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.157203 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157156 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-cni-dir\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.157281 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157212 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffpj9\" (UniqueName: \"kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9\") pod \"network-check-target-mp2lj\" (UID: \"b29e3588-460b-47e5-b419-0af9a3f0f3a5\") " pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:25:57.157281 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157245 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhw7d\" (UniqueName: \"kubernetes.io/projected/3a8ca608-59d8-4480-925d-00d01e6eda10-kube-api-access-rhw7d\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.157363 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157294 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/001a040d-8e63-4eac-8af4-5ce45a0e4914-serviceca\") pod \"node-ca-qjh75\" (UID: \"001a040d-8e63-4eac-8af4-5ce45a0e4914\") " pod="openshift-image-registry/node-ca-qjh75" Jul 10 18:25:57.157363 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157324 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-system-cni-dir\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.157363 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157354 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-etc-kubernetes\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.157479 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-run-openvswitch\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.157479 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157411 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-cni-bin\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.157479 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157442 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.157587 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157485 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-run-ovn\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.157587 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157517 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8feead6f-4012-48ba-88db-2a3e0937d86b-agent-certs\") pod \"konnectivity-agent-f6s8j\" (UID: \"8feead6f-4012-48ba-88db-2a3e0937d86b\") " pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:25:57.157587 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157542 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2ngc\" (UniqueName: \"kubernetes.io/projected/24908795-9d74-4fec-bb0b-859f3b202d44-kube-api-access-z2ngc\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:25:57.157587 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157577 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-kubelet\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.157742 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157609 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-etc-openvswitch\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.157742 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.157742 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157673 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-var-lib-cni-bin\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.157742 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157701 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-conf-dir\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.157742 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157735 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8feead6f-4012-48ba-88db-2a3e0937d86b-konnectivity-ca\") pod \"konnectivity-agent-f6s8j\" (UID: \"8feead6f-4012-48ba-88db-2a3e0937d86b\") " pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:25:57.157942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157771 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.157942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157825 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-run-k8s-cni-cncf-io\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.157942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157862 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-var-lib-kubelet\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.157942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157890 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-slash\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.157942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157921 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-var-lib-openvswitch\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.158143 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157953 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-cni-netd\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.158143 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.157985 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-hostroot\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.158143 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158022 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.158143 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158052 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-run-netns\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.158143 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158084 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-var-lib-cni-multus\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.158143 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158118 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m8t9\" (UniqueName: \"kubernetes.io/projected/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-kube-api-access-4m8t9\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.158425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158155 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgqkp\" (UniqueName: \"kubernetes.io/projected/001a040d-8e63-4eac-8af4-5ce45a0e4914-kube-api-access-mgqkp\") pod \"node-ca-qjh75\" (UID: \"001a040d-8e63-4eac-8af4-5ce45a0e4914\") " pod="openshift-image-registry/node-ca-qjh75" Jul 10 18:25:57.158425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158188 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-os-release\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.158425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158223 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwrr2\" (UniqueName: \"kubernetes.io/projected/15c9edb2-53dc-4360-a559-1e478b3f98f2-kube-api-access-fwrr2\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.158425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158263 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.158425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158322 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/836709d7-ffc4-4b5a-92ca-f3bfbc4709b8-iptables-alerter-script\") pod \"iptables-alerter-wm6v8\" (UID: \"836709d7-ffc4-4b5a-92ca-f3bfbc4709b8\") " pod="openshift-network-operator/iptables-alerter-wm6v8" Jul 10 18:25:57.158425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158360 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-daemon-config\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.158425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158395 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/836709d7-ffc4-4b5a-92ca-f3bfbc4709b8-host-slash\") pod \"iptables-alerter-wm6v8\" (UID: \"836709d7-ffc4-4b5a-92ca-f3bfbc4709b8\") " pod="openshift-network-operator/iptables-alerter-wm6v8" Jul 10 18:25:57.158754 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158438 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3a8ca608-59d8-4480-925d-00d01e6eda10-ovnkube-script-lib\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.158754 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158466 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/098c1a69-137c-45c4-a356-ed61de961a17-kubelet-config\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:57.158754 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158502 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-cnibin\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.158754 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158563 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15c9edb2-53dc-4360-a559-1e478b3f98f2-cni-binary-copy\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.158754 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.158620 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:25:57.159404 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159299 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m467m\" (UniqueName: \"kubernetes.io/projected/836709d7-ffc4-4b5a-92ca-f3bfbc4709b8-kube-api-access-m467m\") pod \"iptables-alerter-wm6v8\" (UID: \"836709d7-ffc4-4b5a-92ca-f3bfbc4709b8\") " pod="openshift-network-operator/iptables-alerter-wm6v8" Jul 10 18:25:57.159404 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159349 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-run-netns\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.159404 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159398 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:57.159601 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159429 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/001a040d-8e63-4eac-8af4-5ce45a0e4914-host\") pod \"node-ca-qjh75\" (UID: \"001a040d-8e63-4eac-8af4-5ce45a0e4914\") " pod="openshift-image-registry/node-ca-qjh75" Jul 10 18:25:57.159601 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159461 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-systemd-units\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.159601 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159516 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-run-systemd\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.159601 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159563 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.159807 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159613 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3a8ca608-59d8-4480-925d-00d01e6eda10-env-overrides\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.159807 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159650 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-socket-dir-parent\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.159807 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159701 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-node-log\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.159807 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159748 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-cnibin\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.159807 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159796 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-run-multus-certs\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.160042 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.159832 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3a8ca608-59d8-4480-925d-00d01e6eda10-ovnkube-config\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.160097 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.160074 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-67vwq\"" Jul 10 18:25:57.160315 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.160289 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 18:25:57.160393 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.160328 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:25:57.162912 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.162856 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m6n6f\"" Jul 10 18:25:57.162912 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.162871 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 18:25:57.162912 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.162856 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 18:25:57.163200 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.163179 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:25:57.182074 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.182047 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gzh8q" Jul 10 18:25:57.188074 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.187981 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gzh8q" Jul 10 18:25:57.246709 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.246677 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 18:25:57.260137 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260102 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/098c1a69-137c-45c4-a356-ed61de961a17-kubelet-config\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:57.260337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260151 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-sysctl-d\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.260337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260177 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-kubelet-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.260337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260220 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-cnibin\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.260337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260241 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/098c1a69-137c-45c4-a356-ed61de961a17-kubelet-config\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:57.260337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260247 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15c9edb2-53dc-4360-a559-1e478b3f98f2-cni-binary-copy\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.260337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:25:57.260337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260328 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m467m\" (UniqueName: \"kubernetes.io/projected/836709d7-ffc4-4b5a-92ca-f3bfbc4709b8-kube-api-access-m467m\") pod \"iptables-alerter-wm6v8\" (UID: \"836709d7-ffc4-4b5a-92ca-f3bfbc4709b8\") " pod="openshift-network-operator/iptables-alerter-wm6v8" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260352 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-run-netns\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260378 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260403 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-kubernetes\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260427 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/001a040d-8e63-4eac-8af4-5ce45a0e4914-host\") pod \"node-ca-qjh75\" (UID: \"001a040d-8e63-4eac-8af4-5ce45a0e4914\") " pod="openshift-image-registry/node-ca-qjh75" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260451 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-systemd-units\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260475 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-run-systemd\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260501 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260527 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3a8ca608-59d8-4480-925d-00d01e6eda10-env-overrides\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260553 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-tmp\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260581 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-socket-dir-parent\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260609 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-node-log\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260650 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-etc-selinux\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260676 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-cnibin\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260701 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-run-multus-certs\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.260717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3a8ca608-59d8-4480-925d-00d01e6eda10-ovnkube-config\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260750 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-sys\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260783 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-os-release\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260846 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-log-socket\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260872 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a8ca608-59d8-4480-925d-00d01e6eda10-ovn-node-metrics-cert\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260897 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/098c1a69-137c-45c4-a356-ed61de961a17-dbus\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260921 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-lib-modules\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260945 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-var-lib-kubelet\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.260985 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-host\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261091 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-log-socket\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261130 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-run-systemd\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261132 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-os-release\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261155 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-cnibin\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261174 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261190 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-socket-dir-parent\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.261293 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:25:57.261504 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.261365 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs podName:24908795-9d74-4fec-bb0b-859f3b202d44 nodeName:}" failed. No retries permitted until 2026-07-10 18:25:57.761347865 +0000 UTC m=+2.967187899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs") pod "network-metrics-daemon-tvbm5" (UID: "24908795-9d74-4fec-bb0b-859f3b202d44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261552 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-system-cni-dir\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261575 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-cni-dir\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261596 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpj9\" (UniqueName: \"kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9\") pod \"network-check-target-mp2lj\" (UID: \"b29e3588-460b-47e5-b419-0af9a3f0f3a5\") " pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261614 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhw7d\" (UniqueName: \"kubernetes.io/projected/3a8ca608-59d8-4480-925d-00d01e6eda10-kube-api-access-rhw7d\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261635 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-modprobe-d\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261652 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-socket-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261668 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5cfq\" (UniqueName: \"kubernetes.io/projected/9f3d26f3-a321-481c-94c7-4862eeacf579-kube-api-access-r5cfq\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261686 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/001a040d-8e63-4eac-8af4-5ce45a0e4914-serviceca\") pod \"node-ca-qjh75\" (UID: \"001a040d-8e63-4eac-8af4-5ce45a0e4914\") " pod="openshift-image-registry/node-ca-qjh75" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261704 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-system-cni-dir\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261719 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-etc-kubernetes\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261735 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-run-openvswitch\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261752 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-cni-bin\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261761 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3a8ca608-59d8-4480-925d-00d01e6eda10-env-overrides\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261786 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-registration-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261830 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-node-log\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.262327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261857 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-run-netns\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261870 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-systemd-units\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261857 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-cnibin\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261895 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/001a040d-8e63-4eac-8af4-5ce45a0e4914-host\") pod \"node-ca-qjh75\" (UID: \"001a040d-8e63-4eac-8af4-5ce45a0e4914\") " pod="openshift-image-registry/node-ca-qjh75" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261921 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-run-multus-certs\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.261956 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262002 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-cni-dir\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.261552 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15c9edb2-53dc-4360-a559-1e478b3f98f2-cni-binary-copy\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.262021 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret podName:098c1a69-137c-45c4-a356-ed61de961a17 nodeName:}" failed. No retries permitted until 2026-07-10 18:25:57.762003239 +0000 UTC m=+2.967843280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret") pod "global-pull-secret-syncer-8kx2z" (UID: "098c1a69-137c-45c4-a356-ed61de961a17") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262066 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-system-cni-dir\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262420 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/001a040d-8e63-4eac-8af4-5ce45a0e4914-serviceca\") pod \"node-ca-qjh75\" (UID: \"001a040d-8e63-4eac-8af4-5ce45a0e4914\") " pod="openshift-image-registry/node-ca-qjh75" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262425 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3a8ca608-59d8-4480-925d-00d01e6eda10-ovnkube-config\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262470 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-system-cni-dir\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262497 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-etc-kubernetes\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262529 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-run-openvswitch\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262557 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-cni-bin\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262576 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/098c1a69-137c-45c4-a356-ed61de961a17-dbus\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262583 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.263170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262615 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-run-ovn\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262651 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-run-ovn\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262679 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8feead6f-4012-48ba-88db-2a3e0937d86b-agent-certs\") pod \"konnectivity-agent-f6s8j\" (UID: \"8feead6f-4012-48ba-88db-2a3e0937d86b\") " pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262710 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2ngc\" (UniqueName: \"kubernetes.io/projected/24908795-9d74-4fec-bb0b-859f3b202d44-kube-api-access-z2ngc\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262731 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-kubelet\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262741 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-etc-openvswitch\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262829 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262854 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-var-lib-cni-bin\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262921 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-conf-dir\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262950 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8feead6f-4012-48ba-88db-2a3e0937d86b-konnectivity-ca\") pod \"konnectivity-agent-f6s8j\" (UID: \"8feead6f-4012-48ba-88db-2a3e0937d86b\") " pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.262983 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263013 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-run-k8s-cni-cncf-io\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263049 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-var-lib-kubelet\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263105 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-conf-dir\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263104 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-etc-openvswitch\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263146 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-var-lib-cni-bin\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263148 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-slash\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.264095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-var-lib-openvswitch\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263203 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-slash\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263214 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-cni-netd\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263298 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-kubelet\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263343 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694-tmp-dir\") pod \"node-resolver-7bh5c\" (UID: \"f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694\") " pod="openshift-dns/node-resolver-7bh5c" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263366 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-hostroot\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263383 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-run\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263403 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-tuned\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263426 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-var-lib-kubelet\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263449 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263469 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-run-netns\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263479 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-var-lib-openvswitch\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263497 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-var-lib-cni-multus\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263524 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3a8ca608-59d8-4480-925d-00d01e6eda10-host-cni-netd\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263527 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b4bb\" (UniqueName: \"kubernetes.io/projected/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-kube-api-access-9b4bb\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263567 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-sys-fs\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263577 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-hostroot\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.265012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263599 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8d72\" (UniqueName: \"kubernetes.io/projected/f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694-kube-api-access-k8d72\") pod \"node-resolver-7bh5c\" (UID: \"f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694\") " pod="openshift-dns/node-resolver-7bh5c" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263629 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-var-lib-cni-multus\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263636 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m8t9\" (UniqueName: \"kubernetes.io/projected/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-kube-api-access-4m8t9\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263667 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgqkp\" (UniqueName: \"kubernetes.io/projected/001a040d-8e63-4eac-8af4-5ce45a0e4914-kube-api-access-mgqkp\") pod \"node-ca-qjh75\" (UID: \"001a040d-8e63-4eac-8af4-5ce45a0e4914\") " pod="openshift-image-registry/node-ca-qjh75" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263602 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8feead6f-4012-48ba-88db-2a3e0937d86b-konnectivity-ca\") pod \"konnectivity-agent-f6s8j\" (UID: \"8feead6f-4012-48ba-88db-2a3e0937d86b\") " pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263723 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-os-release\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263753 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwrr2\" (UniqueName: \"kubernetes.io/projected/15c9edb2-53dc-4360-a559-1e478b3f98f2-kube-api-access-fwrr2\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263777 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-run-k8s-cni-cncf-io\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263784 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-sysconfig\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263757 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263808 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263902 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263934 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-systemd\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.263997 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-host-run-netns\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264030 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15c9edb2-53dc-4360-a559-1e478b3f98f2-os-release\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-device-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264150 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.265934 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264200 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/836709d7-ffc4-4b5a-92ca-f3bfbc4709b8-iptables-alerter-script\") pod \"iptables-alerter-wm6v8\" (UID: \"836709d7-ffc4-4b5a-92ca-f3bfbc4709b8\") " pod="openshift-network-operator/iptables-alerter-wm6v8" Jul 10 18:25:57.266794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264259 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-sysctl-conf\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.266794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264312 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694-hosts-file\") pod \"node-resolver-7bh5c\" (UID: \"f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694\") " pod="openshift-dns/node-resolver-7bh5c" Jul 10 18:25:57.266794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264349 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-daemon-config\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.266794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264379 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/836709d7-ffc4-4b5a-92ca-f3bfbc4709b8-host-slash\") pod \"iptables-alerter-wm6v8\" (UID: \"836709d7-ffc4-4b5a-92ca-f3bfbc4709b8\") " pod="openshift-network-operator/iptables-alerter-wm6v8" Jul 10 18:25:57.266794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264406 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3a8ca608-59d8-4480-925d-00d01e6eda10-ovnkube-script-lib\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.266794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264627 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/836709d7-ffc4-4b5a-92ca-f3bfbc4709b8-host-slash\") pod \"iptables-alerter-wm6v8\" (UID: \"836709d7-ffc4-4b5a-92ca-f3bfbc4709b8\") " pod="openshift-network-operator/iptables-alerter-wm6v8" Jul 10 18:25:57.266794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264715 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.266794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264841 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/836709d7-ffc4-4b5a-92ca-f3bfbc4709b8-iptables-alerter-script\") pod \"iptables-alerter-wm6v8\" (UID: \"836709d7-ffc4-4b5a-92ca-f3bfbc4709b8\") " pod="openshift-network-operator/iptables-alerter-wm6v8" Jul 10 18:25:57.266794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264939 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/15c9edb2-53dc-4360-a559-1e478b3f98f2-multus-daemon-config\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.266794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.264946 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3a8ca608-59d8-4480-925d-00d01e6eda10-ovnkube-script-lib\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.267949 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.267925 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a8ca608-59d8-4480-925d-00d01e6eda10-ovn-node-metrics-cert\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.268910 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.268881 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8feead6f-4012-48ba-88db-2a3e0937d86b-agent-certs\") pod \"konnectivity-agent-f6s8j\" (UID: \"8feead6f-4012-48ba-88db-2a3e0937d86b\") " pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:25:57.272755 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.272729 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:25:57.272856 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.272761 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:25:57.272856 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.272777 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ffpj9 for pod openshift-network-diagnostics/network-check-target-mp2lj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:25:57.272856 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.272838 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9 podName:b29e3588-460b-47e5-b419-0af9a3f0f3a5 nodeName:}" failed. No retries permitted until 2026-07-10 18:25:57.772816567 +0000 UTC m=+2.978656604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ffpj9" (UniqueName: "kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9") pod "network-check-target-mp2lj" (UID: "b29e3588-460b-47e5-b419-0af9a3f0f3a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:25:57.274858 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.274751 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m467m\" (UniqueName: \"kubernetes.io/projected/836709d7-ffc4-4b5a-92ca-f3bfbc4709b8-kube-api-access-m467m\") pod \"iptables-alerter-wm6v8\" (UID: \"836709d7-ffc4-4b5a-92ca-f3bfbc4709b8\") " pod="openshift-network-operator/iptables-alerter-wm6v8" Jul 10 18:25:57.275817 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.275790 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgqkp\" (UniqueName: \"kubernetes.io/projected/001a040d-8e63-4eac-8af4-5ce45a0e4914-kube-api-access-mgqkp\") pod \"node-ca-qjh75\" (UID: \"001a040d-8e63-4eac-8af4-5ce45a0e4914\") " pod="openshift-image-registry/node-ca-qjh75" Jul 10 18:25:57.276527 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.276504 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m8t9\" (UniqueName: \"kubernetes.io/projected/ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5-kube-api-access-4m8t9\") pod \"multus-additional-cni-plugins-vdnl4\" (UID: \"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5\") " pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.277359 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.277316 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2ngc\" (UniqueName: \"kubernetes.io/projected/24908795-9d74-4fec-bb0b-859f3b202d44-kube-api-access-z2ngc\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:25:57.278323 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.278259 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhw7d\" (UniqueName: \"kubernetes.io/projected/3a8ca608-59d8-4480-925d-00d01e6eda10-kube-api-access-rhw7d\") pod \"ovnkube-node-4ft2m\" (UID: \"3a8ca608-59d8-4480-925d-00d01e6eda10\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.280155 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.280111 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwrr2\" (UniqueName: \"kubernetes.io/projected/15c9edb2-53dc-4360-a559-1e478b3f98f2-kube-api-access-fwrr2\") pod \"multus-q8vll\" (UID: \"15c9edb2-53dc-4360-a559-1e478b3f98f2\") " pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.293867 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.293768 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" event={"ID":"8c74eff82b7e1b9da47b5565ae9756ea","Type":"ContainerStarted","Data":"292e38f8b9eace2170495a5c66dd92b0a436ef3384e52f6657e889163e501984"} Jul 10 18:25:57.295457 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.295398 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-224.ec2.internal" event={"ID":"9979441dfb575ea6a2636357cc791013","Type":"ContainerStarted","Data":"a0cfd10b2f92e8d83bbc8fc5d0b9f1e34f9d83c4624f40f124ece2055424e1c9"} Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365078 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694-hosts-file\") pod \"node-resolver-7bh5c\" (UID: \"f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694\") " pod="openshift-dns/node-resolver-7bh5c" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365164 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-sysctl-d\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-kubelet-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365324 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-kubernetes\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365387 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-tmp\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365425 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-etc-selinux\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365481 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-sys\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365515 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-lib-modules\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365544 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-var-lib-kubelet\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365572 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-host\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365634 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-modprobe-d\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365665 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-socket-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365691 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5cfq\" (UniqueName: \"kubernetes.io/projected/9f3d26f3-a321-481c-94c7-4862eeacf579-kube-api-access-r5cfq\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365732 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-registration-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365779 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694-tmp-dir\") pod \"node-resolver-7bh5c\" (UID: \"f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694\") " pod="openshift-dns/node-resolver-7bh5c" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365810 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-run\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.366197 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365867 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-tuned\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365898 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b4bb\" (UniqueName: \"kubernetes.io/projected/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-kube-api-access-9b4bb\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365955 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-sys-fs\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.365986 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8d72\" (UniqueName: \"kubernetes.io/projected/f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694-kube-api-access-k8d72\") pod \"node-resolver-7bh5c\" (UID: \"f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694\") " pod="openshift-dns/node-resolver-7bh5c" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.366047 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-sysconfig\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.366220 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-systemd\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.366300 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-device-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.366333 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-sysctl-conf\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.366679 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-sysconfig\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.366684 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-registration-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.366684 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-device-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.366724 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-sysctl-conf\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.366735 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-systemd\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.366874 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-socket-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.366966 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-sys-fs\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367034 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-sys\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367057 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694-tmp-dir\") pod \"node-resolver-7bh5c\" (UID: \"f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694\") " pod="openshift-dns/node-resolver-7bh5c" Jul 10 18:25:57.367228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367079 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-etc-selinux\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.368135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367129 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9f3d26f3-a321-481c-94c7-4862eeacf579-kubelet-dir\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.368135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367128 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-lib-modules\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.368135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367162 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-kubernetes\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.368135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367192 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-run\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.368135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367223 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-host\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.368135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367343 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694-hosts-file\") pod \"node-resolver-7bh5c\" (UID: \"f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694\") " pod="openshift-dns/node-resolver-7bh5c" Jul 10 18:25:57.368135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367350 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-sysctl-d\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.368135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367386 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-var-lib-kubelet\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.368135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.367527 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-modprobe-d\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.370613 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.370583 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-etc-tuned\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.370613 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.370602 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-tmp\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.376071 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.376021 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b4bb\" (UniqueName: \"kubernetes.io/projected/0ecf287c-4891-4678-9d8d-20f9dc06d0f5-kube-api-access-9b4bb\") pod \"tuned-hck9f\" (UID: \"0ecf287c-4891-4678-9d8d-20f9dc06d0f5\") " pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.376829 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.376780 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8d72\" (UniqueName: \"kubernetes.io/projected/f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694-kube-api-access-k8d72\") pod \"node-resolver-7bh5c\" (UID: \"f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694\") " pod="openshift-dns/node-resolver-7bh5c" Jul 10 18:25:57.378597 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.378561 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5cfq\" (UniqueName: \"kubernetes.io/projected/9f3d26f3-a321-481c-94c7-4862eeacf579-kube-api-access-r5cfq\") pod \"aws-ebs-csi-driver-node-25d9g\" (UID: \"9f3d26f3-a321-481c-94c7-4862eeacf579\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.438734 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.438695 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vdnl4" Jul 10 18:25:57.448241 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.447912 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qjh75" Jul 10 18:25:57.455235 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:57.455195 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf79a51_c7d4_47a8_8a87_8d6ccd4e90d5.slice/crio-2dbadc6d0b5d4187ca3fce01bce307265a959729cfb9302231aecd25a90729ee WatchSource:0}: Error finding container 2dbadc6d0b5d4187ca3fce01bce307265a959729cfb9302231aecd25a90729ee: Status 404 returned error can't find the container with id 2dbadc6d0b5d4187ca3fce01bce307265a959729cfb9302231aecd25a90729ee Jul 10 18:25:57.458835 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.458659 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q8vll" Jul 10 18:25:57.466191 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.466154 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wm6v8" Jul 10 18:25:57.472484 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:57.472455 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c9edb2_53dc_4360_a559_1e478b3f98f2.slice/crio-31b2cf08a29434f5b4d2410fa882aaa1409f2bd54f4a482b50565d02b9b8ed90 WatchSource:0}: Error finding container 31b2cf08a29434f5b4d2410fa882aaa1409f2bd54f4a482b50565d02b9b8ed90: Status 404 returned error can't find the container with id 31b2cf08a29434f5b4d2410fa882aaa1409f2bd54f4a482b50565d02b9b8ed90 Jul 10 18:25:57.476659 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.476635 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:25:57.479585 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:57.479504 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod836709d7_ffc4_4b5a_92ca_f3bfbc4709b8.slice/crio-0f25e8fa29f323b2430b1da8cb98a5c531c203ad719338c8ebfd18371a543a69 WatchSource:0}: Error finding container 0f25e8fa29f323b2430b1da8cb98a5c531c203ad719338c8ebfd18371a543a69: Status 404 returned error can't find the container with id 0f25e8fa29f323b2430b1da8cb98a5c531c203ad719338c8ebfd18371a543a69 Jul 10 18:25:57.484446 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.484425 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:25:57.489051 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:57.489024 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a8ca608_59d8_4480_925d_00d01e6eda10.slice/crio-20f24502e31f0e440b8b2da976c7d27ee3a5aeac5e646878a6a8115405566520 WatchSource:0}: Error finding container 20f24502e31f0e440b8b2da976c7d27ee3a5aeac5e646878a6a8115405566520: Status 404 returned error can't find the container with id 20f24502e31f0e440b8b2da976c7d27ee3a5aeac5e646878a6a8115405566520 Jul 10 18:25:57.493034 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.493009 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" Jul 10 18:25:57.498951 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:57.498925 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8feead6f_4012_48ba_88db_2a3e0937d86b.slice/crio-313786921ca5a53ac9e97f785fe1ff535b6f7d75f44e80e4be43ce25fe60b017 WatchSource:0}: Error finding container 313786921ca5a53ac9e97f785fe1ff535b6f7d75f44e80e4be43ce25fe60b017: Status 404 returned error can't find the container with id 313786921ca5a53ac9e97f785fe1ff535b6f7d75f44e80e4be43ce25fe60b017 Jul 10 18:25:57.504400 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.504375 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hck9f" Jul 10 18:25:57.511377 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.511353 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7bh5c" Jul 10 18:25:57.523581 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:57.523555 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ecf287c_4891_4678_9d8d_20f9dc06d0f5.slice/crio-fee4a8c39eec3d8fb80eb2d307e6417d8de54b7e8b54f6c0d0d840b234f0f602 WatchSource:0}: Error finding container fee4a8c39eec3d8fb80eb2d307e6417d8de54b7e8b54f6c0d0d840b234f0f602: Status 404 returned error can't find the container with id fee4a8c39eec3d8fb80eb2d307e6417d8de54b7e8b54f6c0d0d840b234f0f602 Jul 10 18:25:57.524823 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:25:57.524798 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d9f5e5_cd7b_4641_b584_1bb2dc6c9694.slice/crio-63a282ac954ebfae2e8a56097c92184b7e4dc120dac1e9445f62031ba3761269 WatchSource:0}: Error finding container 63a282ac954ebfae2e8a56097c92184b7e4dc120dac1e9445f62031ba3761269: Status 404 returned error can't find the container with id 63a282ac954ebfae2e8a56097c92184b7e4dc120dac1e9445f62031ba3761269 Jul 10 18:25:57.770500 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.770408 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:25:57.770500 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.770454 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:57.770735 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.770538 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:25:57.770735 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.770561 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:25:57.770735 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.770626 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs podName:24908795-9d74-4fec-bb0b-859f3b202d44 nodeName:}" failed. No retries permitted until 2026-07-10 18:25:58.770605298 +0000 UTC m=+3.976445332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs") pod "network-metrics-daemon-tvbm5" (UID: "24908795-9d74-4fec-bb0b-859f3b202d44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:25:57.770735 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.770650 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret podName:098c1a69-137c-45c4-a356-ed61de961a17 nodeName:}" failed. No retries permitted until 2026-07-10 18:25:58.77063879 +0000 UTC m=+3.976478826 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret") pod "global-pull-secret-syncer-8kx2z" (UID: "098c1a69-137c-45c4-a356-ed61de961a17") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:25:57.871061 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:57.871021 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpj9\" (UniqueName: \"kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9\") pod \"network-check-target-mp2lj\" (UID: \"b29e3588-460b-47e5-b419-0af9a3f0f3a5\") " pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:25:57.871281 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.871167 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:25:57.871281 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.871183 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:25:57.871281 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.871192 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ffpj9 for pod openshift-network-diagnostics/network-check-target-mp2lj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:25:57.871281 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:57.871275 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9 podName:b29e3588-460b-47e5-b419-0af9a3f0f3a5 nodeName:}" failed. No retries permitted until 2026-07-10 18:25:58.871254779 +0000 UTC m=+4.077094817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ffpj9" (UniqueName: "kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9") pod "network-check-target-mp2lj" (UID: "b29e3588-460b-47e5-b419-0af9a3f0f3a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:25:58.191362 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.191249 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:20:57 +0000 UTC" deadline="2028-04-15 20:19:50.334369992 +0000 UTC" Jul 10 18:25:58.191362 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.191300 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15481h53m52.143076095s" Jul 10 18:25:58.289448 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.289399 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:58.289641 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:58.289534 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:25:58.289641 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.289599 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:25:58.289767 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:58.289736 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:25:58.301586 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.301550 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" event={"ID":"9f3d26f3-a321-481c-94c7-4862eeacf579","Type":"ContainerStarted","Data":"7d5b23643211a97e7cb34980d193ac319a2d6f5ec926e0f681584bad7c8964a8"} Jul 10 18:25:58.304396 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.304354 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" event={"ID":"3a8ca608-59d8-4480-925d-00d01e6eda10","Type":"ContainerStarted","Data":"20f24502e31f0e440b8b2da976c7d27ee3a5aeac5e646878a6a8115405566520"} Jul 10 18:25:58.307327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.307295 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wm6v8" event={"ID":"836709d7-ffc4-4b5a-92ca-f3bfbc4709b8","Type":"ContainerStarted","Data":"0f25e8fa29f323b2430b1da8cb98a5c531c203ad719338c8ebfd18371a543a69"} Jul 10 18:25:58.310053 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.310028 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdnl4" event={"ID":"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5","Type":"ContainerStarted","Data":"2dbadc6d0b5d4187ca3fce01bce307265a959729cfb9302231aecd25a90729ee"} Jul 10 18:25:58.312177 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.312151 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7bh5c" event={"ID":"f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694","Type":"ContainerStarted","Data":"63a282ac954ebfae2e8a56097c92184b7e4dc120dac1e9445f62031ba3761269"} Jul 10 18:25:58.313956 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.313923 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hck9f" event={"ID":"0ecf287c-4891-4678-9d8d-20f9dc06d0f5","Type":"ContainerStarted","Data":"fee4a8c39eec3d8fb80eb2d307e6417d8de54b7e8b54f6c0d0d840b234f0f602"} Jul 10 18:25:58.315877 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.315846 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f6s8j" event={"ID":"8feead6f-4012-48ba-88db-2a3e0937d86b","Type":"ContainerStarted","Data":"313786921ca5a53ac9e97f785fe1ff535b6f7d75f44e80e4be43ce25fe60b017"} Jul 10 18:25:58.317343 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.317316 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q8vll" event={"ID":"15c9edb2-53dc-4360-a559-1e478b3f98f2","Type":"ContainerStarted","Data":"31b2cf08a29434f5b4d2410fa882aaa1409f2bd54f4a482b50565d02b9b8ed90"} Jul 10 18:25:58.319130 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.319106 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qjh75" event={"ID":"001a040d-8e63-4eac-8af4-5ce45a0e4914","Type":"ContainerStarted","Data":"7f2240ac2400e776bea5b6a7fee2adf15b9e2a863c3f1c652ec457252b10b595"} Jul 10 18:25:58.779706 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.778874 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:25:58.779706 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.778931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:25:58.779706 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:58.779100 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:25:58.779706 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:58.779170 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret podName:098c1a69-137c-45c4-a356-ed61de961a17 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:00.77914997 +0000 UTC m=+5.984990007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret") pod "global-pull-secret-syncer-8kx2z" (UID: "098c1a69-137c-45c4-a356-ed61de961a17") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:25:58.779706 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:58.779612 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:25:58.779706 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:58.779668 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs podName:24908795-9d74-4fec-bb0b-859f3b202d44 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:00.779649204 +0000 UTC m=+5.985489242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs") pod "network-metrics-daemon-tvbm5" (UID: "24908795-9d74-4fec-bb0b-859f3b202d44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:25:58.879720 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:58.879678 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpj9\" (UniqueName: \"kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9\") pod \"network-check-target-mp2lj\" (UID: \"b29e3588-460b-47e5-b419-0af9a3f0f3a5\") " pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:25:58.879917 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:58.879894 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:25:58.879917 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:58.879918 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:25:58.880024 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:58.879931 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ffpj9 for pod openshift-network-diagnostics/network-check-target-mp2lj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:25:58.880024 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:58.879994 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9 podName:b29e3588-460b-47e5-b419-0af9a3f0f3a5 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:00.879974658 +0000 UTC m=+6.085814698 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ffpj9" (UniqueName: "kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9") pod "network-check-target-mp2lj" (UID: "b29e3588-460b-47e5-b419-0af9a3f0f3a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:25:59.191645 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:59.191576 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:20:57 +0000 UTC" deadline="2028-03-14 16:22:30.25028768 +0000 UTC" Jul 10 18:25:59.191645 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:59.191614 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14709h56m31.058678347s" Jul 10 18:25:59.289923 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:25:59.289383 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:25:59.289923 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:25:59.289536 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:00.289845 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:00.289078 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:00.289845 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:00.289249 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:00.289845 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:00.289673 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:00.289845 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:00.289791 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:00.796870 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:00.796822 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:00.797061 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:00.796889 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:00.797138 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:00.797064 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:26:00.797138 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:00.797131 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret podName:098c1a69-137c-45c4-a356-ed61de961a17 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:04.797112056 +0000 UTC m=+10.002952096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret") pod "global-pull-secret-syncer-8kx2z" (UID: "098c1a69-137c-45c4-a356-ed61de961a17") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:26:00.797655 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:00.797633 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:26:00.797771 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:00.797699 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs podName:24908795-9d74-4fec-bb0b-859f3b202d44 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:04.797679521 +0000 UTC m=+10.003519575 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs") pod "network-metrics-daemon-tvbm5" (UID: "24908795-9d74-4fec-bb0b-859f3b202d44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:26:00.898987 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:00.898244 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpj9\" (UniqueName: \"kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9\") pod \"network-check-target-mp2lj\" (UID: \"b29e3588-460b-47e5-b419-0af9a3f0f3a5\") " pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:00.898987 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:00.898434 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:26:00.898987 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:00.898460 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:26:00.898987 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:00.898473 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ffpj9 for pod openshift-network-diagnostics/network-check-target-mp2lj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:26:00.898987 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:00.898538 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9 podName:b29e3588-460b-47e5-b419-0af9a3f0f3a5 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:04.898516409 +0000 UTC m=+10.104356450 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ffpj9" (UniqueName: "kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9") pod "network-check-target-mp2lj" (UID: "b29e3588-460b-47e5-b419-0af9a3f0f3a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:26:01.289141 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:01.288559 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:01.289141 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:01.288699 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:02.289399 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:02.289117 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:02.289847 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:02.289528 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:02.289928 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:02.289182 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:02.290023 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:02.289997 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:03.292398 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:03.292354 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:03.292892 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:03.292486 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:04.288514 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:04.288469 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:04.288717 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:04.288616 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:04.289047 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:04.289020 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:04.289146 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:04.289126 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:04.829570 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:04.829484 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:04.829570 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:04.829538 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:04.830070 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:04.829668 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:26:04.830070 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:04.829716 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:26:04.830070 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:04.829754 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs podName:24908795-9d74-4fec-bb0b-859f3b202d44 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:12.829731945 +0000 UTC m=+18.035571999 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs") pod "network-metrics-daemon-tvbm5" (UID: "24908795-9d74-4fec-bb0b-859f3b202d44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:26:04.830070 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:04.829780 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret podName:098c1a69-137c-45c4-a356-ed61de961a17 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:12.829762533 +0000 UTC m=+18.035602565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret") pod "global-pull-secret-syncer-8kx2z" (UID: "098c1a69-137c-45c4-a356-ed61de961a17") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:26:04.930355 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:04.930302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpj9\" (UniqueName: \"kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9\") pod \"network-check-target-mp2lj\" (UID: \"b29e3588-460b-47e5-b419-0af9a3f0f3a5\") " pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:04.930549 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:04.930526 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:26:04.930549 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:04.930544 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:26:04.930689 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:04.930556 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ffpj9 for pod openshift-network-diagnostics/network-check-target-mp2lj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:26:04.930689 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:04.930613 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9 podName:b29e3588-460b-47e5-b419-0af9a3f0f3a5 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:12.93059587 +0000 UTC m=+18.136435905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ffpj9" (UniqueName: "kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9") pod "network-check-target-mp2lj" (UID: "b29e3588-460b-47e5-b419-0af9a3f0f3a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:26:05.289826 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:05.289741 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:05.289991 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:05.289864 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:06.289273 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:06.289239 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:06.289273 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:06.289260 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:06.289788 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:06.289383 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:06.289788 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:06.289514 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:07.289624 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:07.289256 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:07.289624 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:07.289391 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:08.289230 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:08.289180 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:08.289414 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:08.289180 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:08.289414 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:08.289347 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:08.289526 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:08.289481 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:09.292572 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:09.292439 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:09.292572 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:09.292564 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:10.288858 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:10.288818 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:10.289031 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:10.288823 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:10.289031 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:10.288945 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:10.289133 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:10.289023 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:11.288362 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:11.288328 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:11.288732 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:11.288442 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:12.288348 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:12.288313 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:12.288348 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:12.288333 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:12.288865 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:12.288435 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:12.288865 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:12.288580 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:12.889661 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:12.889623 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:12.889856 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:12.889754 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:12.889856 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:12.889765 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:26:12.889856 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:12.889839 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs podName:24908795-9d74-4fec-bb0b-859f3b202d44 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:28.88981638 +0000 UTC m=+34.095656416 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs") pod "network-metrics-daemon-tvbm5" (UID: "24908795-9d74-4fec-bb0b-859f3b202d44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:26:12.890036 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:12.889902 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:26:12.890036 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:12.889956 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret podName:098c1a69-137c-45c4-a356-ed61de961a17 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:28.889939862 +0000 UTC m=+34.095779899 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret") pod "global-pull-secret-syncer-8kx2z" (UID: "098c1a69-137c-45c4-a356-ed61de961a17") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:26:12.990974 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:12.990912 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpj9\" (UniqueName: \"kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9\") pod \"network-check-target-mp2lj\" (UID: \"b29e3588-460b-47e5-b419-0af9a3f0f3a5\") " pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:12.991154 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:12.991095 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:26:12.991154 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:12.991119 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:26:12.991154 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:12.991133 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ffpj9 for pod openshift-network-diagnostics/network-check-target-mp2lj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:26:12.991343 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:12.991211 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9 podName:b29e3588-460b-47e5-b419-0af9a3f0f3a5 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:28.991177004 +0000 UTC m=+34.197017057 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ffpj9" (UniqueName: "kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9") pod "network-check-target-mp2lj" (UID: "b29e3588-460b-47e5-b419-0af9a3f0f3a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:26:13.289261 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:13.289092 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:13.289261 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:13.289248 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:14.288770 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:14.288713 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:14.288958 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:14.288772 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:14.288958 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:14.288924 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:14.289073 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:14.289015 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:15.288960 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:15.288927 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:15.289467 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:15.289038 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:16.288441 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:16.288414 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:16.288526 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:16.288422 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:16.288557 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:16.288519 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:16.288622 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:16.288605 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:17.289337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.289091 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:17.290264 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:17.289433 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:17.370838 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.370804 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hck9f" event={"ID":"0ecf287c-4891-4678-9d8d-20f9dc06d0f5","Type":"ContainerStarted","Data":"795b749387991cc21c84e1b4d028c567452996e6585594a26b277fd9e2573d6e"} Jul 10 18:26:17.372453 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.372420 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q8vll" event={"ID":"15c9edb2-53dc-4360-a559-1e478b3f98f2","Type":"ContainerStarted","Data":"e3eeb89181edd995500c7f5ece3b269ff085d29fb2b7fa4f57b4c9a39199885c"} Jul 10 18:26:17.373900 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.373877 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-224.ec2.internal" event={"ID":"9979441dfb575ea6a2636357cc791013","Type":"ContainerStarted","Data":"061b2ac288590b0165c475743aa0e6a014cc8d8779df6fe26832561767e465a0"} Jul 10 18:26:17.377119 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.377102 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:26:17.377444 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.377426 2565 generic.go:356] "Generic (PLEG): container finished" podID="3a8ca608-59d8-4480-925d-00d01e6eda10" containerID="3f7a9c12c3c7fabd06ad5512c46e833b1d71eb4fec6d18840427eb9a802f2acf" exitCode=1 Jul 10 18:26:17.377508 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.377455 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" event={"ID":"3a8ca608-59d8-4480-925d-00d01e6eda10","Type":"ContainerStarted","Data":"88dd889bdc2e86ebf9ddb7a8bd55121d043dad3f2f414adb09d433b064df0dba"} Jul 10 18:26:17.377508 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.377503 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" event={"ID":"3a8ca608-59d8-4480-925d-00d01e6eda10","Type":"ContainerStarted","Data":"ffd369bafeac004d3cb9d506a231deb11be87caa71e618162feebf0c78b35315"} Jul 10 18:26:17.377581 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.377534 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" event={"ID":"3a8ca608-59d8-4480-925d-00d01e6eda10","Type":"ContainerStarted","Data":"770dc7c49b76d0799f24e62e3203000284225544b59a2b2db5e4b48d2ded7b0c"} Jul 10 18:26:17.377581 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.377562 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" event={"ID":"3a8ca608-59d8-4480-925d-00d01e6eda10","Type":"ContainerStarted","Data":"9df3dfbf3f6e34cfc311c3c2a9be564dcd991771239d045d43378435ef147bc4"} Jul 10 18:26:17.377648 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.377587 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" event={"ID":"3a8ca608-59d8-4480-925d-00d01e6eda10","Type":"ContainerDied","Data":"3f7a9c12c3c7fabd06ad5512c46e833b1d71eb4fec6d18840427eb9a802f2acf"} Jul 10 18:26:17.377648 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.377615 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" event={"ID":"3a8ca608-59d8-4480-925d-00d01e6eda10","Type":"ContainerStarted","Data":"9cb1be72969e959f439e7c7ed45a85accba7900ae2699f46939472135b1deb0d"} Jul 10 18:26:17.388669 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.388630 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hck9f" podStartSLOduration=2.434427013 podStartE2EDuration="21.388618175s" totalImagesPullingTime="18.954191162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:17.388090598 +0000 UTC m=+22.593930654" watchObservedRunningTime="2026-07-10 18:26:17.388618175 +0000 UTC m=+22.594458229" Jul 10 18:26:17.401621 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.401585 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-224.ec2.internal" podStartSLOduration=21.401573062 podStartE2EDuration="21.401573062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:17.401332443 +0000 UTC m=+22.607172498" watchObservedRunningTime="2026-07-10 18:26:17.401573062 +0000 UTC m=+22.607413117" Jul 10 18:26:17.426000 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:17.425958 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q8vll" podStartSLOduration=2.400501965 podStartE2EDuration="21.425945871s" totalImagesPullingTime="19.025443906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:17.425836092 +0000 UTC m=+22.631676148" watchObservedRunningTime="2026-07-10 18:26:17.425945871 +0000 UTC m=+22.631786001" Jul 10 18:26:18.289456 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.289223 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:18.289972 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.289223 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:18.289972 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:18.289560 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:18.289972 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:18.289668 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:18.381349 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.381318 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f6s8j" event={"ID":"8feead6f-4012-48ba-88db-2a3e0937d86b","Type":"ContainerStarted","Data":"6abfa659f27df30b65394f38b2c51bc42fc4ee5d1e26745755a972a417413729"} Jul 10 18:26:18.382704 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.382680 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qjh75" event={"ID":"001a040d-8e63-4eac-8af4-5ce45a0e4914","Type":"ContainerStarted","Data":"668c5cb20ebd29265e6ff5d3bb85ea6cdf6e5962f319b28f67b544c33701c748"} Jul 10 18:26:18.384088 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.384062 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" event={"ID":"9f3d26f3-a321-481c-94c7-4862eeacf579","Type":"ContainerStarted","Data":"059bc16beb8b495db86c6a996def522e21838e1c5b40ac95db7a0f593c950987"} Jul 10 18:26:18.385449 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.385427 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wm6v8" event={"ID":"836709d7-ffc4-4b5a-92ca-f3bfbc4709b8","Type":"ContainerStarted","Data":"0d1469f42082d061fdadcd75a80e6d73b34b9de314668909e013a277d503982d"} Jul 10 18:26:18.386756 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.386732 2565 generic.go:356] "Generic (PLEG): container finished" podID="ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5" containerID="4f109b51e3c86b28d11d008ece9a2f4aa3f36d3a2fc11fac301fe499d89e873e" exitCode=0 Jul 10 18:26:18.386835 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.386799 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdnl4" event={"ID":"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5","Type":"ContainerDied","Data":"4f109b51e3c86b28d11d008ece9a2f4aa3f36d3a2fc11fac301fe499d89e873e"} Jul 10 18:26:18.388267 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.388245 2565 generic.go:356] "Generic (PLEG): container finished" podID="8c74eff82b7e1b9da47b5565ae9756ea" containerID="23585c5c5351a2c777bf4897c611ae2b17771b8ecfea765ed053aedb390b5b6e" exitCode=0 Jul 10 18:26:18.388341 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.388299 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" event={"ID":"8c74eff82b7e1b9da47b5565ae9756ea","Type":"ContainerDied","Data":"23585c5c5351a2c777bf4897c611ae2b17771b8ecfea765ed053aedb390b5b6e"} Jul 10 18:26:18.389806 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.389786 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7bh5c" event={"ID":"f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694","Type":"ContainerStarted","Data":"0f80bed5180719e8da4b68ca17801ef1138bff82b3060350494fc587d98fb41b"} Jul 10 18:26:18.396370 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.396331 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-f6s8j" podStartSLOduration=3.421792298 podStartE2EDuration="22.396318575s" totalImagesPullingTime="18.974526277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:18.395869158 +0000 UTC m=+23.601709213" watchObservedRunningTime="2026-07-10 18:26:18.396318575 +0000 UTC m=+23.602158629" Jul 10 18:26:18.410255 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.410216 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7bh5c" podStartSLOduration=3.460130974 podStartE2EDuration="22.410186822s" totalImagesPullingTime="18.950055848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:18.410063243 +0000 UTC m=+23.615903318" watchObservedRunningTime="2026-07-10 18:26:18.410186822 +0000 UTC m=+23.616026877" Jul 10 18:26:18.459595 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.459537 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wm6v8" podStartSLOduration=3.464297701 podStartE2EDuration="22.459518674s" totalImagesPullingTime="18.995220973s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:18.45903244 +0000 UTC m=+23.664872506" watchObservedRunningTime="2026-07-10 18:26:18.459518674 +0000 UTC m=+23.665358730" Jul 10 18:26:18.473516 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.473450 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qjh75" podStartSLOduration=11.89604517 podStartE2EDuration="22.473431353s" totalImagesPullingTime="10.577386183s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:18.472990343 +0000 UTC m=+23.678830418" watchObservedRunningTime="2026-07-10 18:26:18.473431353 +0000 UTC m=+23.679271405" Jul 10 18:26:18.893632 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:18.893609 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 18:26:19.231763 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:19.231592 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T18:26:18.893626673Z","UUID":"96a21c62-f8f1-4138-a39d-031aab3e34fb","Handler":null,"Name":"","Endpoint":""} Jul 10 18:26:19.233174 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:19.233153 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 18:26:19.233300 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:19.233207 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 18:26:19.289003 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:19.288970 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:19.289237 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:19.289110 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:19.393901 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:19.393857 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" event={"ID":"8c74eff82b7e1b9da47b5565ae9756ea","Type":"ContainerStarted","Data":"f1a8de86722191f64efce48335775364d5deb099f4f26ac5acc0a91c8c177a1c"} Jul 10 18:26:19.395816 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:19.395785 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" event={"ID":"9f3d26f3-a321-481c-94c7-4862eeacf579","Type":"ContainerStarted","Data":"5755c9fc8cc35006cdf6c4822a88f362771b8901834d02a31bc705b138706f8b"} Jul 10 18:26:19.409619 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:19.409573 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-224.ec2.internal" podStartSLOduration=23.409558422 podStartE2EDuration="23.409558422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:19.409438519 +0000 UTC m=+24.615278573" watchObservedRunningTime="2026-07-10 18:26:19.409558422 +0000 UTC m=+24.615398476" Jul 10 18:26:20.289031 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:20.289000 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:20.289031 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:20.289042 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:20.289261 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:20.289128 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:20.289296 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:20.289270 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:20.399520 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:20.399483 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" event={"ID":"9f3d26f3-a321-481c-94c7-4862eeacf579","Type":"ContainerStarted","Data":"32a269453907054cc97e2dfffc565dab20b0fcc682f4b8f28d5ecf184e312b2c"} Jul 10 18:26:20.402426 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:20.402406 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:26:20.402717 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:20.402694 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" event={"ID":"3a8ca608-59d8-4480-925d-00d01e6eda10","Type":"ContainerStarted","Data":"dc6203395dfed9d90969a37dfd9627870239605e2a60d3e1a4fc1bb861da4ca2"} Jul 10 18:26:20.421509 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:20.421465 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25d9g" podStartSLOduration=11.373941009 podStartE2EDuration="24.42145022s" totalImagesPullingTime="13.047509211s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:20.420983909 +0000 UTC m=+25.626823964" watchObservedRunningTime="2026-07-10 18:26:20.42145022 +0000 UTC m=+25.627290274" Jul 10 18:26:21.289210 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:21.289167 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:21.289381 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:21.289319 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:21.821320 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:21.821058 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:26:21.822035 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:21.821805 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:26:22.289318 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.289245 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:22.289318 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.289290 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:22.289515 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:22.289388 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:22.289572 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:22.289539 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:22.410363 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.410333 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:26:22.410800 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.410771 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" event={"ID":"3a8ca608-59d8-4480-925d-00d01e6eda10","Type":"ContainerStarted","Data":"d6987bad16efb57241cdfdd4c9503104670a92ddee68a8174b5a4a41a6bfa585"} Jul 10 18:26:22.411394 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.411162 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:26:22.411394 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.411254 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:26:22.411394 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.411288 2565 scope.go:117] "RemoveContainer" containerID="3f7a9c12c3c7fabd06ad5512c46e833b1d71eb4fec6d18840427eb9a802f2acf" Jul 10 18:26:22.411394 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.411300 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:26:22.411394 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.411346 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:26:22.411807 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.411788 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-f6s8j" Jul 10 18:26:22.428570 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.428546 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:26:22.428956 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:22.428932 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:26:23.289316 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:23.289288 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:23.289847 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:23.289409 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:23.781655 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:23.781487 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tvbm5"] Jul 10 18:26:23.781791 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:23.781771 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:23.781899 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:23.781879 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:23.785072 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:23.785044 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mp2lj"] Jul 10 18:26:23.785222 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:23.785152 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:23.785303 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:23.785277 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:23.785744 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:23.785684 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8kx2z"] Jul 10 18:26:23.785872 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:23.785796 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:23.785936 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:23.785893 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:24.416703 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:24.416667 2565 generic.go:356] "Generic (PLEG): container finished" podID="ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5" containerID="4be6cb899e16119253a4e3628f338e12cf530df749d7680d156434cb67592ddb" exitCode=0 Jul 10 18:26:24.417112 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:24.416761 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdnl4" event={"ID":"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5","Type":"ContainerDied","Data":"4be6cb899e16119253a4e3628f338e12cf530df749d7680d156434cb67592ddb"} Jul 10 18:26:24.424016 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:24.423995 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:26:24.424378 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:24.424349 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" event={"ID":"3a8ca608-59d8-4480-925d-00d01e6eda10","Type":"ContainerStarted","Data":"f4638f53114d72fee359d9410e1c63809f99ca88a3ec829a0c0ebd001063a3c4"} Jul 10 18:26:24.465146 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:24.465089 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" podStartSLOduration=9.427506321 podStartE2EDuration="28.465073801s" totalImagesPullingTime="19.03756748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:24.464907586 +0000 UTC m=+29.670747640" watchObservedRunningTime="2026-07-10 18:26:24.465073801 +0000 UTC m=+29.670913856" Jul 10 18:26:25.289489 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:25.289454 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:25.289666 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:25.289561 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:25.289666 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:25.289575 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:25.289666 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:25.289598 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:25.289815 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:25.289689 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:25.289815 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:25.289777 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:25.428318 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:25.428240 2565 generic.go:356] "Generic (PLEG): container finished" podID="ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5" containerID="1f4783a22403597aecda0ab14c4eb6d2fdbd34f175b5b74318450f58de5cf8ee" exitCode=0 Jul 10 18:26:25.428661 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:25.428318 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdnl4" event={"ID":"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5","Type":"ContainerDied","Data":"1f4783a22403597aecda0ab14c4eb6d2fdbd34f175b5b74318450f58de5cf8ee"} Jul 10 18:26:26.432814 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:26.432732 2565 generic.go:356] "Generic (PLEG): container finished" podID="ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5" containerID="70ef59c03cd015dcae6dbd83b688cd152fb85a46a310588d828542a4ce832b8a" exitCode=0 Jul 10 18:26:26.432814 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:26.432789 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdnl4" event={"ID":"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5","Type":"ContainerDied","Data":"70ef59c03cd015dcae6dbd83b688cd152fb85a46a310588d828542a4ce832b8a"} Jul 10 18:26:27.289067 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:27.289029 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:27.289254 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:27.289078 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:27.289254 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:27.289030 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:27.289254 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:27.289172 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:27.289440 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:27.289272 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:27.289440 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:27.289371 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:28.905646 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:28.905412 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:28.906181 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:28.905669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:28.906181 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:28.905563 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:26:28.906181 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:28.905786 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:26:28.906181 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:28.905812 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs podName:24908795-9d74-4fec-bb0b-859f3b202d44 nodeName:}" failed. No retries permitted until 2026-07-10 18:27:00.905784294 +0000 UTC m=+66.111624339 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs") pod "network-metrics-daemon-tvbm5" (UID: "24908795-9d74-4fec-bb0b-859f3b202d44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:26:28.906181 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:28.905833 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret podName:098c1a69-137c-45c4-a356-ed61de961a17 nodeName:}" failed. No retries permitted until 2026-07-10 18:27:00.905821846 +0000 UTC m=+66.111661879 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret") pod "global-pull-secret-syncer-8kx2z" (UID: "098c1a69-137c-45c4-a356-ed61de961a17") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:26:29.007034 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.006990 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpj9\" (UniqueName: \"kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9\") pod \"network-check-target-mp2lj\" (UID: \"b29e3588-460b-47e5-b419-0af9a3f0f3a5\") " pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:29.007236 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.007142 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:26:29.007236 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.007161 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:26:29.007236 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.007175 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ffpj9 for pod openshift-network-diagnostics/network-check-target-mp2lj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:26:29.007402 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.007253 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9 podName:b29e3588-460b-47e5-b419-0af9a3f0f3a5 nodeName:}" failed. No retries permitted until 2026-07-10 18:27:01.007236639 +0000 UTC m=+66.213076694 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ffpj9" (UniqueName: "kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9") pod "network-check-target-mp2lj" (UID: "b29e3588-460b-47e5-b419-0af9a3f0f3a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:26:29.288564 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.288482 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:29.288564 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.288503 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:29.288759 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.288480 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:29.288759 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.288627 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mp2lj" podUID="b29e3588-460b-47e5-b419-0af9a3f0f3a5" Jul 10 18:26:29.288854 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.288759 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:26:29.288914 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.288867 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8kx2z" podUID="098c1a69-137c-45c4-a356-ed61de961a17" Jul 10 18:26:29.576383 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.576311 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-224.ec2.internal" event="NodeReady" Jul 10 18:26:29.576546 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.576464 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 18:26:29.610093 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.610058 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vt5c2"] Jul 10 18:26:29.653148 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.653116 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67b79b8bcc-fwqbw"] Jul 10 18:26:29.653332 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.653269 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:26:29.656268 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.656236 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 10 18:26:29.656268 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.656242 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 10 18:26:29.656441 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.656293 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-lkrwm\"" Jul 10 18:26:29.666797 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.666766 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vt5c2"] Jul 10 18:26:29.666936 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.666813 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5ft29"] Jul 10 18:26:29.666936 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.666886 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.670040 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.669849 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 18:26:29.670040 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.669935 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rc4ck\"" Jul 10 18:26:29.670040 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.669941 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 18:26:29.670040 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.669984 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 18:26:29.675123 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.675103 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 18:26:29.684943 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.684918 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67b79b8bcc-fwqbw"] Jul 10 18:26:29.685047 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.684947 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-psq6h"] Jul 10 18:26:29.685047 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.685034 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.687873 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.687827 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 18:26:29.687975 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.687873 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 18:26:29.688163 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.688147 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-z2lbt\"" Jul 10 18:26:29.710575 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.710535 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5ft29"] Jul 10 18:26:29.710575 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.710580 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-psq6h"] Jul 10 18:26:29.710770 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.710670 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-psq6h" Jul 10 18:26:29.713517 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.713492 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 18:26:29.713667 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.713562 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 18:26:29.713667 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.713587 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 18:26:29.713667 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.713586 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9bsdn\"" Jul 10 18:26:29.813995 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.813958 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-trusted-ca\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.814181 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814005 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06300a4b-433b-4fa0-b969-78d216179a3b-tmp-dir\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.814181 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814026 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-image-registry-private-configuration\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.814181 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814052 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnwp4\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-kube-api-access-mnwp4\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.814181 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814076 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5ff392a-52f5-4430-9b45-c07ecc3445ec-cert\") pod \"ingress-canary-psq6h\" (UID: \"c5ff392a-52f5-4430-9b45-c07ecc3445ec\") " pod="openshift-ingress-canary/ingress-canary-psq6h" Jul 10 18:26:29.814181 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814137 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.814459 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814225 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk7gk\" (UniqueName: \"kubernetes.io/projected/c5ff392a-52f5-4430-9b45-c07ecc3445ec-kube-api-access-qk7gk\") pod \"ingress-canary-psq6h\" (UID: \"c5ff392a-52f5-4430-9b45-c07ecc3445ec\") " pod="openshift-ingress-canary/ingress-canary-psq6h" Jul 10 18:26:29.814459 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814265 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-installation-pull-secrets\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.814459 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814321 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-certificates\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.814459 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814346 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/06300a4b-433b-4fa0-b969-78d216179a3b-config-volume\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.814459 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814369 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bqqt\" (UniqueName: \"kubernetes.io/projected/06300a4b-433b-4fa0-b969-78d216179a3b-kube-api-access-4bqqt\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.814459 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814386 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-bound-sa-token\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.814459 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814415 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:26:29.814459 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814445 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.814790 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814469 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:26:29.814790 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.814488 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f08196c-7a64-43e1-9dd4-bf32a34b2844-ca-trust-extracted\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.915537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915444 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-trusted-ca\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.915537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915497 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06300a4b-433b-4fa0-b969-78d216179a3b-tmp-dir\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915555 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-image-registry-private-configuration\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915576 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnwp4\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-kube-api-access-mnwp4\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915599 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5ff392a-52f5-4430-9b45-c07ecc3445ec-cert\") pod \"ingress-canary-psq6h\" (UID: \"c5ff392a-52f5-4430-9b45-c07ecc3445ec\") " pod="openshift-ingress-canary/ingress-canary-psq6h" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915627 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915653 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk7gk\" (UniqueName: \"kubernetes.io/projected/c5ff392a-52f5-4430-9b45-c07ecc3445ec-kube-api-access-qk7gk\") pod \"ingress-canary-psq6h\" (UID: \"c5ff392a-52f5-4430-9b45-c07ecc3445ec\") " pod="openshift-ingress-canary/ingress-canary-psq6h" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915685 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-installation-pull-secrets\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915737 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-certificates\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915759 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/06300a4b-433b-4fa0-b969-78d216179a3b-config-volume\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915783 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqqt\" (UniqueName: \"kubernetes.io/projected/06300a4b-433b-4fa0-b969-78d216179a3b-kube-api-access-4bqqt\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915808 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-bound-sa-token\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915847 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915872 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915894 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:26:29.916030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.915914 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f08196c-7a64-43e1-9dd4-bf32a34b2844-ca-trust-extracted\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.916734 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.916247 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06300a4b-433b-4fa0-b969-78d216179a3b-tmp-dir\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.916734 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.916359 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f08196c-7a64-43e1-9dd4-bf32a34b2844-ca-trust-extracted\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.916734 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.916440 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:26:29.916734 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.916454 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b79b8bcc-fwqbw: secret "image-registry-tls" not found Jul 10 18:26:29.916734 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.916505 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls podName:2f08196c-7a64-43e1-9dd4-bf32a34b2844 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:30.416487664 +0000 UTC m=+35.622327702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls") pod "image-registry-67b79b8bcc-fwqbw" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844") : secret "image-registry-tls" not found Jul 10 18:26:29.916734 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.916554 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:26:29.916734 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.916571 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/06300a4b-433b-4fa0-b969-78d216179a3b-config-volume\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.916734 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.916593 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert podName:5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac nodeName:}" failed. No retries permitted until 2026-07-10 18:26:30.416581359 +0000 UTC m=+35.622421406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vt5c2" (UID: "5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac") : secret "networking-console-plugin-cert" not found Jul 10 18:26:29.916734 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.916652 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:26:29.916734 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.916670 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:26:29.917232 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:29.916806 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls podName:06300a4b-433b-4fa0-b969-78d216179a3b nodeName:}" failed. No retries permitted until 2026-07-10 18:26:30.416787331 +0000 UTC m=+35.622627382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls") pod "dns-default-5ft29" (UID: "06300a4b-433b-4fa0-b969-78d216179a3b") : secret "dns-default-metrics-tls" not found Jul 10 18:26:29.917232 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.916907 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-trusted-ca\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.917338 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.917237 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-certificates\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.920611 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.920590 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-installation-pull-secrets\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.920747 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.920614 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5ff392a-52f5-4430-9b45-c07ecc3445ec-cert\") pod \"ingress-canary-psq6h\" (UID: \"c5ff392a-52f5-4430-9b45-c07ecc3445ec\") " pod="openshift-ingress-canary/ingress-canary-psq6h" Jul 10 18:26:29.920936 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.920917 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-image-registry-private-configuration\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.926012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.925981 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bqqt\" (UniqueName: \"kubernetes.io/projected/06300a4b-433b-4fa0-b969-78d216179a3b-kube-api-access-4bqqt\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:29.926261 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.926184 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnwp4\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-kube-api-access-mnwp4\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.926582 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.926563 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-bound-sa-token\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:29.926668 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:29.926601 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk7gk\" (UniqueName: \"kubernetes.io/projected/c5ff392a-52f5-4430-9b45-c07ecc3445ec-kube-api-access-qk7gk\") pod \"ingress-canary-psq6h\" (UID: \"c5ff392a-52f5-4430-9b45-c07ecc3445ec\") " pod="openshift-ingress-canary/ingress-canary-psq6h" Jul 10 18:26:30.021463 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:30.021427 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-psq6h" Jul 10 18:26:30.420879 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:30.420842 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:30.421036 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:30.420967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:30.421036 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:30.421002 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:26:30.421156 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:30.421025 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:26:30.421156 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:30.421110 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:26:30.421156 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:30.421116 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:26:30.421156 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:30.421127 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls podName:06300a4b-433b-4fa0-b969-78d216179a3b nodeName:}" failed. No retries permitted until 2026-07-10 18:26:31.421103348 +0000 UTC m=+36.626943385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls") pod "dns-default-5ft29" (UID: "06300a4b-433b-4fa0-b969-78d216179a3b") : secret "dns-default-metrics-tls" not found Jul 10 18:26:30.421156 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:30.421137 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b79b8bcc-fwqbw: secret "image-registry-tls" not found Jul 10 18:26:30.421386 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:30.421166 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert podName:5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac nodeName:}" failed. No retries permitted until 2026-07-10 18:26:31.421148267 +0000 UTC m=+36.626988306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vt5c2" (UID: "5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac") : secret "networking-console-plugin-cert" not found Jul 10 18:26:30.421386 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:30.421246 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls podName:2f08196c-7a64-43e1-9dd4-bf32a34b2844 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:31.421228525 +0000 UTC m=+36.627068578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls") pod "image-registry-67b79b8bcc-fwqbw" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844") : secret "image-registry-tls" not found Jul 10 18:26:30.579780 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:30.579749 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-psq6h"] Jul 10 18:26:31.289318 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.289280 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:26:31.290150 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.289282 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:26:31.290150 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.289432 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:26:31.293859 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.293832 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hnv2c\"" Jul 10 18:26:31.293859 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.293832 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:26:31.294059 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.293906 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 18:26:31.294059 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.293833 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 18:26:31.294059 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.293833 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fnqlf\"" Jul 10 18:26:31.294059 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.293833 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:26:31.430479 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.430440 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:31.430666 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:31.430614 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:26:31.430737 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:31.430701 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls podName:06300a4b-433b-4fa0-b969-78d216179a3b nodeName:}" failed. No retries permitted until 2026-07-10 18:26:33.430678008 +0000 UTC m=+38.636518060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls") pod "dns-default-5ft29" (UID: "06300a4b-433b-4fa0-b969-78d216179a3b") : secret "dns-default-metrics-tls" not found Jul 10 18:26:31.430803 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.430752 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:31.430803 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.430787 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:26:31.430930 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:31.430907 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:26:31.430930 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:31.430931 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b79b8bcc-fwqbw: secret "image-registry-tls" not found Jul 10 18:26:31.431061 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:31.430984 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls podName:2f08196c-7a64-43e1-9dd4-bf32a34b2844 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:33.430967986 +0000 UTC m=+38.636808030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls") pod "image-registry-67b79b8bcc-fwqbw" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844") : secret "image-registry-tls" not found Jul 10 18:26:31.431061 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:31.430915 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:26:31.431061 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:31.431051 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert podName:5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac nodeName:}" failed. No retries permitted until 2026-07-10 18:26:33.431034486 +0000 UTC m=+38.636874535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vt5c2" (UID: "5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac") : secret "networking-console-plugin-cert" not found Jul 10 18:26:31.447766 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.447730 2565 generic.go:356] "Generic (PLEG): container finished" podID="ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5" containerID="de86bb9d41930eebe188c0b1215c2f0c9530f1c284143ea2d8aa55cfbb2be952" exitCode=0 Jul 10 18:26:31.447930 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.447818 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdnl4" event={"ID":"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5","Type":"ContainerDied","Data":"de86bb9d41930eebe188c0b1215c2f0c9530f1c284143ea2d8aa55cfbb2be952"} Jul 10 18:26:31.449327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:31.449302 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-psq6h" event={"ID":"c5ff392a-52f5-4430-9b45-c07ecc3445ec","Type":"ContainerStarted","Data":"82540cfa728f53f20ba6f35bac9fd2804e7151aa8faccb2254e87a2cc734a3f0"} Jul 10 18:26:32.455592 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:32.455552 2565 generic.go:356] "Generic (PLEG): container finished" podID="ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5" containerID="846e8f832c98fda12f7a9c094ae3ba62ccb8ff832d1166aa560046492df868db" exitCode=0 Jul 10 18:26:32.456026 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:32.455607 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdnl4" event={"ID":"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5","Type":"ContainerDied","Data":"846e8f832c98fda12f7a9c094ae3ba62ccb8ff832d1166aa560046492df868db"} Jul 10 18:26:33.447680 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:33.447590 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:33.447814 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:33.447690 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:33.447814 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:33.447720 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:26:33.447814 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:33.447743 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:26:33.447918 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:33.447816 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls podName:06300a4b-433b-4fa0-b969-78d216179a3b nodeName:}" failed. No retries permitted until 2026-07-10 18:26:37.447799038 +0000 UTC m=+42.653639079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls") pod "dns-default-5ft29" (UID: "06300a4b-433b-4fa0-b969-78d216179a3b") : secret "dns-default-metrics-tls" not found Jul 10 18:26:33.447918 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:33.447819 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:26:33.447918 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:33.447837 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b79b8bcc-fwqbw: secret "image-registry-tls" not found Jul 10 18:26:33.447918 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:33.447816 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:26:33.447918 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:33.447877 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls podName:2f08196c-7a64-43e1-9dd4-bf32a34b2844 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:37.447864168 +0000 UTC m=+42.653704200 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls") pod "image-registry-67b79b8bcc-fwqbw" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844") : secret "image-registry-tls" not found Jul 10 18:26:33.448073 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:33.447934 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert podName:5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac nodeName:}" failed. No retries permitted until 2026-07-10 18:26:37.447919203 +0000 UTC m=+42.653759249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vt5c2" (UID: "5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac") : secret "networking-console-plugin-cert" not found Jul 10 18:26:33.460911 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:33.460874 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vdnl4" event={"ID":"ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5","Type":"ContainerStarted","Data":"f318c4d07489db93a9a772aed79e2972ce1f7eb22ff704271c9d8440ad7bee9a"} Jul 10 18:26:33.484567 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:33.484519 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vdnl4" podStartSLOduration=7.494013383 podStartE2EDuration="37.484501548s" totalImagesPullingTime="29.990488165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:33.482703702 +0000 UTC m=+38.688543756" watchObservedRunningTime="2026-07-10 18:26:33.484501548 +0000 UTC m=+38.690341603" Jul 10 18:26:33.627544 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:33.627514 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:33.887437 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:33.887404 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:34.060644 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:34.060605 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:34.121778 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:34.121743 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:34.464389 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:34.464345 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-psq6h" event={"ID":"c5ff392a-52f5-4430-9b45-c07ecc3445ec","Type":"ContainerStarted","Data":"6cda7985253ab13605d26df87e551fd691c5868a3c8be92ad0081cca19adb31e"} Jul 10 18:26:34.481023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:34.480982 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-psq6h" podStartSLOduration=2.619040746 podStartE2EDuration="5.48096682s" totalImagesPullingTime="2.861926074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:34.480472781 +0000 UTC m=+39.686312863" watchObservedRunningTime="2026-07-10 18:26:34.48096682 +0000 UTC m=+39.686806912" Jul 10 18:26:34.797948 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:34.797875 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:34.871940 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:34.871906 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:37.480383 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:37.480347 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:37.480383 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:37.480386 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:26:37.480789 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:37.480413 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:37.480789 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:37.480513 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:26:37.480789 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:37.480593 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert podName:5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac nodeName:}" failed. No retries permitted until 2026-07-10 18:26:45.480578041 +0000 UTC m=+50.686418077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vt5c2" (UID: "5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac") : secret "networking-console-plugin-cert" not found Jul 10 18:26:37.480789 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:37.480516 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:26:37.480789 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:37.480620 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b79b8bcc-fwqbw: secret "image-registry-tls" not found Jul 10 18:26:37.480789 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:37.480669 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls podName:2f08196c-7a64-43e1-9dd4-bf32a34b2844 nodeName:}" failed. No retries permitted until 2026-07-10 18:26:45.480655246 +0000 UTC m=+50.686495279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls") pod "image-registry-67b79b8bcc-fwqbw" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844") : secret "image-registry-tls" not found Jul 10 18:26:37.480789 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:37.480524 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:26:37.480789 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:37.480712 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls podName:06300a4b-433b-4fa0-b969-78d216179a3b nodeName:}" failed. No retries permitted until 2026-07-10 18:26:45.480702883 +0000 UTC m=+50.686542917 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls") pod "dns-default-5ft29" (UID: "06300a4b-433b-4fa0-b969-78d216179a3b") : secret "dns-default-metrics-tls" not found Jul 10 18:26:42.667896 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:42.667852 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:42.766152 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:42.766104 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:43.633809 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:43.633776 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:43.733325 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:43.733290 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:43.932658 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:43.932568 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:44.020023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:44.019983 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:44.136596 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:44.136559 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:45.547762 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:45.547723 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:26:45.547762 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:45.547762 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:26:45.548284 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:45.547804 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:26:45.548284 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:45.547878 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:26:45.548284 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:45.547896 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b79b8bcc-fwqbw: secret "image-registry-tls" not found Jul 10 18:26:45.548284 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:45.547898 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:26:45.548284 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:45.547928 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:26:45.548284 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:45.547949 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls podName:2f08196c-7a64-43e1-9dd4-bf32a34b2844 nodeName:}" failed. No retries permitted until 2026-07-10 18:27:01.547932749 +0000 UTC m=+66.753772782 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls") pod "image-registry-67b79b8bcc-fwqbw" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844") : secret "image-registry-tls" not found Jul 10 18:26:45.548284 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:45.547964 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert podName:5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac nodeName:}" failed. No retries permitted until 2026-07-10 18:27:01.547956371 +0000 UTC m=+66.753796403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vt5c2" (UID: "5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac") : secret "networking-console-plugin-cert" not found Jul 10 18:26:45.548284 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:26:45.547975 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls podName:06300a4b-433b-4fa0-b969-78d216179a3b nodeName:}" failed. No retries permitted until 2026-07-10 18:27:01.547970382 +0000 UTC m=+66.753810414 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls") pod "dns-default-5ft29" (UID: "06300a4b-433b-4fa0-b969-78d216179a3b") : secret "dns-default-metrics-tls" not found Jul 10 18:26:45.945937 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:45.945903 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:46.041729 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:46.041690 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:46.877871 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:46.877839 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:48.877844 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:48.877808 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:55.438516 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:55.438487 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ft2m" Jul 10 18:26:55.638954 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:55.638920 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:55.698102 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:26:55.697997 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:00.519975 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:00.519916 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:00.741613 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:00.741570 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:00.841941 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:00.841836 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:00.944749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:00.944717 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:00.965760 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:00.965733 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:27:00.965928 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:00.965768 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:27:00.969059 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:00.969038 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:27:00.969265 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:00.969249 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:27:00.976825 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:00.976810 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 18:27:00.976909 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:00.976862 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs podName:24908795-9d74-4fec-bb0b-859f3b202d44 nodeName:}" failed. No retries permitted until 2026-07-10 18:28:04.976847301 +0000 UTC m=+130.182687333 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs") pod "network-metrics-daemon-tvbm5" (UID: "24908795-9d74-4fec-bb0b-859f3b202d44") : secret "metrics-daemon-secret" not found Jul 10 18:27:00.980368 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:00.980352 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/098c1a69-137c-45c4-a356-ed61de961a17-original-pull-secret\") pod \"global-pull-secret-syncer-8kx2z\" (UID: \"098c1a69-137c-45c4-a356-ed61de961a17\") " pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:27:01.008566 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.008539 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8kx2z" Jul 10 18:27:01.044561 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.044532 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:01.067012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.066971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpj9\" (UniqueName: \"kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9\") pod \"network-check-target-mp2lj\" (UID: \"b29e3588-460b-47e5-b419-0af9a3f0f3a5\") " pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:27:01.070431 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.070413 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 18:27:01.081495 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.081280 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 18:27:01.104681 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.104566 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffpj9\" (UniqueName: \"kubernetes.io/projected/b29e3588-460b-47e5-b419-0af9a3f0f3a5-kube-api-access-ffpj9\") pod \"network-check-target-mp2lj\" (UID: \"b29e3588-460b-47e5-b419-0af9a3f0f3a5\") " pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:27:01.115048 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.115018 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:01.174661 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.174633 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8kx2z"] Jul 10 18:27:01.178064 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:27:01.178035 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod098c1a69_137c_45c4_a356_ed61de961a17.slice/crio-98d8986662f975b4f92e8fadbb2ccf51cbde65227db78a5dfac90d28b075d331 WatchSource:0}: Error finding container 98d8986662f975b4f92e8fadbb2ccf51cbde65227db78a5dfac90d28b075d331: Status 404 returned error can't find the container with id 98d8986662f975b4f92e8fadbb2ccf51cbde65227db78a5dfac90d28b075d331 Jul 10 18:27:01.318396 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.318369 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hnv2c\"" Jul 10 18:27:01.325864 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.325843 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:27:01.478815 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.478784 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mp2lj"] Jul 10 18:27:01.481614 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:27:01.481591 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb29e3588_460b_47e5_b419_0af9a3f0f3a5.slice/crio-93c93a192200f3641d05498ec39ce7648a8f7037d65b1c8c4051ac51a8cab15c WatchSource:0}: Error finding container 93c93a192200f3641d05498ec39ce7648a8f7037d65b1c8c4051ac51a8cab15c: Status 404 returned error can't find the container with id 93c93a192200f3641d05498ec39ce7648a8f7037d65b1c8c4051ac51a8cab15c Jul 10 18:27:01.531325 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.531289 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8kx2z" event={"ID":"098c1a69-137c-45c4-a356-ed61de961a17","Type":"ContainerStarted","Data":"98d8986662f975b4f92e8fadbb2ccf51cbde65227db78a5dfac90d28b075d331"} Jul 10 18:27:01.532380 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.532356 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mp2lj" event={"ID":"b29e3588-460b-47e5-b419-0af9a3f0f3a5","Type":"ContainerStarted","Data":"93c93a192200f3641d05498ec39ce7648a8f7037d65b1c8c4051ac51a8cab15c"} Jul 10 18:27:01.570856 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.570819 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:27:01.571011 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.570909 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:27:01.571011 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.570939 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:27:01.571011 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:01.570967 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:27:01.571178 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:01.571044 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls podName:06300a4b-433b-4fa0-b969-78d216179a3b nodeName:}" failed. No retries permitted until 2026-07-10 18:27:33.571026518 +0000 UTC m=+98.776866552 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls") pod "dns-default-5ft29" (UID: "06300a4b-433b-4fa0-b969-78d216179a3b") : secret "dns-default-metrics-tls" not found Jul 10 18:27:01.571178 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:01.571045 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:27:01.571178 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:01.571111 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert podName:5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac nodeName:}" failed. No retries permitted until 2026-07-10 18:27:33.571095253 +0000 UTC m=+98.776935289 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vt5c2" (UID: "5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac") : secret "networking-console-plugin-cert" not found Jul 10 18:27:01.571178 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:01.571045 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:27:01.571178 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:01.571134 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b79b8bcc-fwqbw: secret "image-registry-tls" not found Jul 10 18:27:01.571178 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:01.571168 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls podName:2f08196c-7a64-43e1-9dd4-bf32a34b2844 nodeName:}" failed. No retries permitted until 2026-07-10 18:27:33.571158521 +0000 UTC m=+98.776998558 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls") pod "image-registry-67b79b8bcc-fwqbw" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844") : secret "image-registry-tls" not found Jul 10 18:27:01.685986 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.685894 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:01.739756 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:01.739715 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:05.543728 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:05.543697 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mp2lj" event={"ID":"b29e3588-460b-47e5-b419-0af9a3f0f3a5","Type":"ContainerStarted","Data":"3f02b618e430a9756bf381630ffe9dac9d1ecfc77d83c996445e18c9ae80284e"} Jul 10 18:27:05.544113 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:05.543851 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:27:05.562948 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:05.562903 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mp2lj" podStartSLOduration=66.090590138 podStartE2EDuration="1m9.562888747s" totalImagesPullingTime="3.472298609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:27:05.561873663 +0000 UTC m=+70.767713737" watchObservedRunningTime="2026-07-10 18:27:05.562888747 +0000 UTC m=+70.768728801" Jul 10 18:27:07.550511 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:07.550477 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8kx2z" event={"ID":"098c1a69-137c-45c4-a356-ed61de961a17","Type":"ContainerStarted","Data":"f42390071c302c4e0e50f644eb9d991649da10dc6c0b736a8f3c6c63f7851e93"} Jul 10 18:27:07.587514 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:07.587465 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8kx2z" podStartSLOduration=65.662049281 podStartE2EDuration="1m11.587449864s" totalImagesPullingTime="5.925400583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:27:07.587311547 +0000 UTC m=+72.793151603" watchObservedRunningTime="2026-07-10 18:27:07.587449864 +0000 UTC m=+72.793289918" Jul 10 18:27:21.718312 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:21.718171 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:21.774271 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:21.774231 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:21.970391 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:21.970282 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:22.029488 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:22.029448 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:22.088751 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:22.088690 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:33.612105 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:33.612072 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:27:33.612515 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:33.612123 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:27:33.612515 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:33.612170 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:27:33.612515 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:33.612216 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:27:33.612515 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:33.612234 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b79b8bcc-fwqbw: secret "image-registry-tls" not found Jul 10 18:27:33.612515 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:33.612284 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:27:33.612515 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:33.612305 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:27:33.612515 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:33.612287 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls podName:2f08196c-7a64-43e1-9dd4-bf32a34b2844 nodeName:}" failed. No retries permitted until 2026-07-10 18:28:37.612267776 +0000 UTC m=+162.818107809 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls") pod "image-registry-67b79b8bcc-fwqbw" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844") : secret "image-registry-tls" not found Jul 10 18:27:33.612515 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:33.612361 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls podName:06300a4b-433b-4fa0-b969-78d216179a3b nodeName:}" failed. No retries permitted until 2026-07-10 18:28:37.612343817 +0000 UTC m=+162.818183864 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls") pod "dns-default-5ft29" (UID: "06300a4b-433b-4fa0-b969-78d216179a3b") : secret "dns-default-metrics-tls" not found Jul 10 18:27:33.612515 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:27:33.612380 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert podName:5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac nodeName:}" failed. No retries permitted until 2026-07-10 18:28:37.612371474 +0000 UTC m=+162.818211513 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vt5c2" (UID: "5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac") : secret "networking-console-plugin-cert" not found Jul 10 18:27:36.549161 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:36.549130 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mp2lj" Jul 10 18:27:58.256646 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:27:58.256605 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:05.038260 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:28:05.038214 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:28:05.038703 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:05.038373 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 18:28:05.038703 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:05.038452 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs podName:24908795-9d74-4fec-bb0b-859f3b202d44 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:07.038433192 +0000 UTC m=+252.244273226 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs") pod "network-metrics-daemon-tvbm5" (UID: "24908795-9d74-4fec-bb0b-859f3b202d44") : secret "metrics-daemon-secret" not found Jul 10 18:28:32.664430 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:32.664367 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" podUID="5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac" Jul 10 18:28:32.677612 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:32.677570 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" podUID="2f08196c-7a64-43e1-9dd4-bf32a34b2844" Jul 10 18:28:32.694754 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:32.694721 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5ft29" podUID="06300a4b-433b-4fa0-b969-78d216179a3b" Jul 10 18:28:32.747894 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:28:32.747871 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:28:32.747894 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:28:32.747880 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:28:32.748069 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:28:32.747871 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5ft29" Jul 10 18:28:34.301764 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:34.301716 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:28:37.672832 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:28:37.672784 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:28:37.673273 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:28:37.672871 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:28:37.673273 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:28:37.672907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:28:37.673273 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:37.672958 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:28:37.673273 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:37.672990 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 18:28:37.673273 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:37.673009 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:28:37.673273 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:37.673028 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b79b8bcc-fwqbw: secret "image-registry-tls" not found Jul 10 18:28:37.673273 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:37.673045 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls podName:06300a4b-433b-4fa0-b969-78d216179a3b nodeName:}" failed. No retries permitted until 2026-07-10 18:30:39.673026153 +0000 UTC m=+284.878866198 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls") pod "dns-default-5ft29" (UID: "06300a4b-433b-4fa0-b969-78d216179a3b") : secret "dns-default-metrics-tls" not found Jul 10 18:28:37.673273 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:37.673061 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert podName:5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac nodeName:}" failed. No retries permitted until 2026-07-10 18:30:39.673054752 +0000 UTC m=+284.878894787 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vt5c2" (UID: "5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac") : secret "networking-console-plugin-cert" not found Jul 10 18:28:37.673273 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:28:37.673082 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls podName:2f08196c-7a64-43e1-9dd4-bf32a34b2844 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:39.67306626 +0000 UTC m=+284.878906293 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls") pod "image-registry-67b79b8bcc-fwqbw" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844") : secret "image-registry-tls" not found Jul 10 18:28:49.289395 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:28:49.289309 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:29:42.614592 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:29:42.614551 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:07.121830 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:07.121772 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:30:07.122347 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:07.121937 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 18:30:07.122347 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:07.122051 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs podName:24908795-9d74-4fec-bb0b-859f3b202d44 nodeName:}" failed. No retries permitted until 2026-07-10 18:32:09.122018239 +0000 UTC m=+374.327858278 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs") pod "network-metrics-daemon-tvbm5" (UID: "24908795-9d74-4fec-bb0b-859f3b202d44") : secret "metrics-daemon-secret" not found Jul 10 18:30:12.870856 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.867732 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj"] Jul 10 18:30:12.871483 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.871465 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" Jul 10 18:30:12.873911 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.873887 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc"] Jul 10 18:30:12.874591 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.874560 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 10 18:30:12.874723 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.874563 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 10 18:30:12.874723 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.874647 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:30:12.874723 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.874628 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 10 18:30:12.875914 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.875900 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-nbhxl\"" Jul 10 18:30:12.876577 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.876545 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:12.880425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.880406 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms"] Jul 10 18:30:12.881494 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.881473 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 18:30:12.881588 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.881471 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 10 18:30:12.881588 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.881474 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 18:30:12.881588 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.881474 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cql49\"" Jul 10 18:30:12.881716 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.881526 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 10 18:30:12.883023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.883006 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-h5p2f"] Jul 10 18:30:12.883141 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.883118 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" Jul 10 18:30:12.885635 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.885616 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-h5p2f" Jul 10 18:30:12.895504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.895486 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj"] Jul 10 18:30:12.896941 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.896923 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 10 18:30:12.897422 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.897405 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 10 18:30:12.897471 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.897423 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:30:12.897546 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.897533 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-k4grm\"" Jul 10 18:30:12.898033 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.898016 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 10 18:30:12.898187 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.898165 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-gg47n\"" Jul 10 18:30:12.904009 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.903982 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc"] Jul 10 18:30:12.914585 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.914561 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms"] Jul 10 18:30:12.917292 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.917267 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-h5p2f"] Jul 10 18:30:12.963891 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.963863 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d-serving-cert\") pod \"service-ca-operator-7695fc6c8c-25szj\" (UID: \"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" Jul 10 18:30:12.964046 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.963895 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8hfr\" (UniqueName: \"kubernetes.io/projected/d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d-kube-api-access-r8hfr\") pod \"service-ca-operator-7695fc6c8c-25szj\" (UID: \"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" Jul 10 18:30:12.964046 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.963917 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:12.964046 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.963932 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d-config\") pod \"service-ca-operator-7695fc6c8c-25szj\" (UID: \"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" Jul 10 18:30:12.964046 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.964036 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d96h\" (UniqueName: \"kubernetes.io/projected/7cb95732-885e-4683-b8cf-eda8406c7000-kube-api-access-2d96h\") pod \"network-check-source-869dcfcdcd-h5p2f\" (UID: \"7cb95732-885e-4683-b8cf-eda8406c7000\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-h5p2f" Jul 10 18:30:12.964203 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.964061 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b56e500-4923-4ab3-8b46-052e2a42839b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-6kgms\" (UID: \"9b56e500-4923-4ab3-8b46-052e2a42839b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" Jul 10 18:30:12.964203 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.964082 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dbp8\" (UniqueName: \"kubernetes.io/projected/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-kube-api-access-9dbp8\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:12.964203 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.964108 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:12.964203 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.964176 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b56e500-4923-4ab3-8b46-052e2a42839b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-6kgms\" (UID: \"9b56e500-4923-4ab3-8b46-052e2a42839b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" Jul 10 18:30:12.964339 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.964233 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdx9q\" (UniqueName: \"kubernetes.io/projected/9b56e500-4923-4ab3-8b46-052e2a42839b-kube-api-access-hdx9q\") pod \"kube-storage-version-migrator-operator-799cb7f958-6kgms\" (UID: \"9b56e500-4923-4ab3-8b46-052e2a42839b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" Jul 10 18:30:12.979862 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.979835 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kq2db"] Jul 10 18:30:12.982671 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.982656 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-57kb4"] Jul 10 18:30:12.982784 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.982766 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kq2db" Jul 10 18:30:12.985484 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.985465 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:12.985598 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.985488 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 10 18:30:12.985598 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.985541 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:30:12.985902 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.985887 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-d8scx\"" Jul 10 18:30:12.990033 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.990014 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 18:30:12.990184 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.990080 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 10 18:30:12.990314 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.990269 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 10 18:30:12.990412 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.990395 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 18:30:12.990491 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.990440 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pltw5\"" Jul 10 18:30:12.994771 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.994749 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 10 18:30:12.995994 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.995814 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kq2db"] Jul 10 18:30:12.999020 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:12.999000 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-57kb4"] Jul 10 18:30:13.064625 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064582 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b56e500-4923-4ab3-8b46-052e2a42839b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-6kgms\" (UID: \"9b56e500-4923-4ab3-8b46-052e2a42839b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" Jul 10 18:30:13.064802 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064642 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d96h\" (UniqueName: \"kubernetes.io/projected/7cb95732-885e-4683-b8cf-eda8406c7000-kube-api-access-2d96h\") pod \"network-check-source-869dcfcdcd-h5p2f\" (UID: \"7cb95732-885e-4683-b8cf-eda8406c7000\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-h5p2f" Jul 10 18:30:13.064802 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064661 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dbp8\" (UniqueName: \"kubernetes.io/projected/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-kube-api-access-9dbp8\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:13.064802 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064682 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aacd5c76-7b3e-4ad4-817c-279d66dae83e-tmp\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.064802 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064746 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:13.064802 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064780 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b55xr\" (UniqueName: \"kubernetes.io/projected/aacd5c76-7b3e-4ad4-817c-279d66dae83e-kube-api-access-b55xr\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.065125 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064847 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/aacd5c76-7b3e-4ad4-817c-279d66dae83e-snapshots\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.065125 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064888 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8hfr\" (UniqueName: \"kubernetes.io/projected/d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d-kube-api-access-r8hfr\") pod \"service-ca-operator-7695fc6c8c-25szj\" (UID: \"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" Jul 10 18:30:13.065125 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d-config\") pod \"service-ca-operator-7695fc6c8c-25szj\" (UID: \"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" Jul 10 18:30:13.065125 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064940 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aacd5c76-7b3e-4ad4-817c-279d66dae83e-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.065125 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064970 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aacd5c76-7b3e-4ad4-817c-279d66dae83e-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.065125 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.064987 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aacd5c76-7b3e-4ad4-817c-279d66dae83e-serving-cert\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.065125 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.065002 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2wsr\" (UniqueName: \"kubernetes.io/projected/939c9e48-d662-4d9d-96c8-b14367001619-kube-api-access-n2wsr\") pod \"volume-data-source-validator-78495597f6-kq2db\" (UID: \"939c9e48-d662-4d9d-96c8-b14367001619\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kq2db" Jul 10 18:30:13.065125 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.065079 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:13.065538 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.065140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b56e500-4923-4ab3-8b46-052e2a42839b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-6kgms\" (UID: \"9b56e500-4923-4ab3-8b46-052e2a42839b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" Jul 10 18:30:13.065538 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.065178 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdx9q\" (UniqueName: \"kubernetes.io/projected/9b56e500-4923-4ab3-8b46-052e2a42839b-kube-api-access-hdx9q\") pod \"kube-storage-version-migrator-operator-799cb7f958-6kgms\" (UID: \"9b56e500-4923-4ab3-8b46-052e2a42839b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" Jul 10 18:30:13.065538 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.065231 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d-serving-cert\") pod \"service-ca-operator-7695fc6c8c-25szj\" (UID: \"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" Jul 10 18:30:13.065538 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:13.065236 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:13.065538 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:13.065300 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls podName:a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a nodeName:}" failed. No retries permitted until 2026-07-10 18:30:13.565279267 +0000 UTC m=+258.771119320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dw5tc" (UID: "a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:13.065538 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.065329 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b56e500-4923-4ab3-8b46-052e2a42839b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-6kgms\" (UID: \"9b56e500-4923-4ab3-8b46-052e2a42839b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" Jul 10 18:30:13.065726 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.065571 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d-config\") pod \"service-ca-operator-7695fc6c8c-25szj\" (UID: \"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" Jul 10 18:30:13.065726 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.065605 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:13.067408 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.067388 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d-serving-cert\") pod \"service-ca-operator-7695fc6c8c-25szj\" (UID: \"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" Jul 10 18:30:13.067408 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.067398 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b56e500-4923-4ab3-8b46-052e2a42839b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-6kgms\" (UID: \"9b56e500-4923-4ab3-8b46-052e2a42839b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" Jul 10 18:30:13.075253 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.075231 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dbp8\" (UniqueName: \"kubernetes.io/projected/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-kube-api-access-9dbp8\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:13.078027 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.078005 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d96h\" (UniqueName: \"kubernetes.io/projected/7cb95732-885e-4683-b8cf-eda8406c7000-kube-api-access-2d96h\") pod \"network-check-source-869dcfcdcd-h5p2f\" (UID: \"7cb95732-885e-4683-b8cf-eda8406c7000\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-h5p2f" Jul 10 18:30:13.078027 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.078017 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8hfr\" (UniqueName: \"kubernetes.io/projected/d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d-kube-api-access-r8hfr\") pod \"service-ca-operator-7695fc6c8c-25szj\" (UID: \"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" Jul 10 18:30:13.078415 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.078379 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdx9q\" (UniqueName: \"kubernetes.io/projected/9b56e500-4923-4ab3-8b46-052e2a42839b-kube-api-access-hdx9q\") pod \"kube-storage-version-migrator-operator-799cb7f958-6kgms\" (UID: \"9b56e500-4923-4ab3-8b46-052e2a42839b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" Jul 10 18:30:13.116903 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.116868 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:13.166656 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.166569 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b55xr\" (UniqueName: \"kubernetes.io/projected/aacd5c76-7b3e-4ad4-817c-279d66dae83e-kube-api-access-b55xr\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.166656 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.166611 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/aacd5c76-7b3e-4ad4-817c-279d66dae83e-snapshots\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.166656 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.166647 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aacd5c76-7b3e-4ad4-817c-279d66dae83e-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.166924 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.166672 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aacd5c76-7b3e-4ad4-817c-279d66dae83e-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.166924 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.166696 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aacd5c76-7b3e-4ad4-817c-279d66dae83e-serving-cert\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.166924 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.166717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2wsr\" (UniqueName: \"kubernetes.io/projected/939c9e48-d662-4d9d-96c8-b14367001619-kube-api-access-n2wsr\") pod \"volume-data-source-validator-78495597f6-kq2db\" (UID: \"939c9e48-d662-4d9d-96c8-b14367001619\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kq2db" Jul 10 18:30:13.166924 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.166871 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aacd5c76-7b3e-4ad4-817c-279d66dae83e-tmp\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.167285 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.167263 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aacd5c76-7b3e-4ad4-817c-279d66dae83e-tmp\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.167376 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.167354 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/aacd5c76-7b3e-4ad4-817c-279d66dae83e-snapshots\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.167484 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.167469 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aacd5c76-7b3e-4ad4-817c-279d66dae83e-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.167873 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.167855 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aacd5c76-7b3e-4ad4-817c-279d66dae83e-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.168990 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.168972 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aacd5c76-7b3e-4ad4-817c-279d66dae83e-serving-cert\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.175606 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.175571 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b55xr\" (UniqueName: \"kubernetes.io/projected/aacd5c76-7b3e-4ad4-817c-279d66dae83e-kube-api-access-b55xr\") pod \"insights-operator-68d9dd94cf-57kb4\" (UID: \"aacd5c76-7b3e-4ad4-817c-279d66dae83e\") " pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.175716 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.175605 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2wsr\" (UniqueName: \"kubernetes.io/projected/939c9e48-d662-4d9d-96c8-b14367001619-kube-api-access-n2wsr\") pod \"volume-data-source-validator-78495597f6-kq2db\" (UID: \"939c9e48-d662-4d9d-96c8-b14367001619\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kq2db" Jul 10 18:30:13.180301 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.180282 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" Jul 10 18:30:13.193541 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.193516 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" Jul 10 18:30:13.198616 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.198589 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-h5p2f" Jul 10 18:30:13.218738 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.218705 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:13.292335 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.292250 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kq2db" Jul 10 18:30:13.298360 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.297993 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" Jul 10 18:30:13.331332 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.331300 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj"] Jul 10 18:30:13.334744 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:13.334704 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43482b7_b9be_4ecf_b5fb_fe276e9c8a3d.slice/crio-f91bd181b906c9a497e006eaaa21725419997adb0a0dac01c9e1e7b8baf18a68 WatchSource:0}: Error finding container f91bd181b906c9a497e006eaaa21725419997adb0a0dac01c9e1e7b8baf18a68: Status 404 returned error can't find the container with id f91bd181b906c9a497e006eaaa21725419997adb0a0dac01c9e1e7b8baf18a68 Jul 10 18:30:13.350267 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.350229 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms"] Jul 10 18:30:13.360376 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:13.360333 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b56e500_4923_4ab3_8b46_052e2a42839b.slice/crio-92ee82ac39db6465d9e12e0914ab7d7e6669d923a2b0fbc5c1f46534ee51a274 WatchSource:0}: Error finding container 92ee82ac39db6465d9e12e0914ab7d7e6669d923a2b0fbc5c1f46534ee51a274: Status 404 returned error can't find the container with id 92ee82ac39db6465d9e12e0914ab7d7e6669d923a2b0fbc5c1f46534ee51a274 Jul 10 18:30:13.374457 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.373951 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-h5p2f"] Jul 10 18:30:13.380508 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:13.380473 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cb95732_885e_4683_b8cf_eda8406c7000.slice/crio-5f9265836da304581890a9d8070ebf374c1e7dadc219db256ff28fd11540d9c0 WatchSource:0}: Error finding container 5f9265836da304581890a9d8070ebf374c1e7dadc219db256ff28fd11540d9c0: Status 404 returned error can't find the container with id 5f9265836da304581890a9d8070ebf374c1e7dadc219db256ff28fd11540d9c0 Jul 10 18:30:13.450895 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.450751 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kq2db"] Jul 10 18:30:13.453109 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:13.453078 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod939c9e48_d662_4d9d_96c8_b14367001619.slice/crio-646b5769ea096366cc61279cd7676a68e668850ca8a664d6556307f52b01e8b9 WatchSource:0}: Error finding container 646b5769ea096366cc61279cd7676a68e668850ca8a664d6556307f52b01e8b9: Status 404 returned error can't find the container with id 646b5769ea096366cc61279cd7676a68e668850ca8a664d6556307f52b01e8b9 Jul 10 18:30:13.465341 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.465315 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-57kb4"] Jul 10 18:30:13.468818 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:13.468790 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaacd5c76_7b3e_4ad4_817c_279d66dae83e.slice/crio-feec2af20a8c14b073256c46b61cfda2a3e1036b1b67612b48ad37f771b12b5f WatchSource:0}: Error finding container feec2af20a8c14b073256c46b61cfda2a3e1036b1b67612b48ad37f771b12b5f: Status 404 returned error can't find the container with id feec2af20a8c14b073256c46b61cfda2a3e1036b1b67612b48ad37f771b12b5f Jul 10 18:30:13.570872 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.570825 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:13.571112 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:13.570974 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:13.571112 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:13.571043 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls podName:a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a nodeName:}" failed. No retries permitted until 2026-07-10 18:30:14.571019199 +0000 UTC m=+259.776859250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dw5tc" (UID: "a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:13.990804 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.990526 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" event={"ID":"aacd5c76-7b3e-4ad4-817c-279d66dae83e","Type":"ContainerStarted","Data":"feec2af20a8c14b073256c46b61cfda2a3e1036b1b67612b48ad37f771b12b5f"} Jul 10 18:30:13.992298 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.992186 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" event={"ID":"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d","Type":"ContainerStarted","Data":"f91bd181b906c9a497e006eaaa21725419997adb0a0dac01c9e1e7b8baf18a68"} Jul 10 18:30:13.998538 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.997769 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-h5p2f" event={"ID":"7cb95732-885e-4683-b8cf-eda8406c7000","Type":"ContainerStarted","Data":"6adb346daaeca7bb2b5e94f243ce3268eabc6d9c97d5b660ee84281a60f70eb8"} Jul 10 18:30:13.998538 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.997821 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-h5p2f" event={"ID":"7cb95732-885e-4683-b8cf-eda8406c7000","Type":"ContainerStarted","Data":"5f9265836da304581890a9d8070ebf374c1e7dadc219db256ff28fd11540d9c0"} Jul 10 18:30:14.000029 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:13.999999 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kq2db" event={"ID":"939c9e48-d662-4d9d-96c8-b14367001619","Type":"ContainerStarted","Data":"646b5769ea096366cc61279cd7676a68e668850ca8a664d6556307f52b01e8b9"} Jul 10 18:30:14.002931 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:14.002875 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" event={"ID":"9b56e500-4923-4ab3-8b46-052e2a42839b","Type":"ContainerStarted","Data":"92ee82ac39db6465d9e12e0914ab7d7e6669d923a2b0fbc5c1f46534ee51a274"} Jul 10 18:30:14.580040 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:14.579854 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:14.580249 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:14.580117 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:14.580249 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:14.580215 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls podName:a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a nodeName:}" failed. No retries permitted until 2026-07-10 18:30:16.580172967 +0000 UTC m=+261.786013003 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dw5tc" (UID: "a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:15.009681 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:15.009642 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kq2db" event={"ID":"939c9e48-d662-4d9d-96c8-b14367001619","Type":"ContainerStarted","Data":"cfd0728eb88f25c1ad5504089f50bcbb22244536c29f6a67a6c04920859cba1c"} Jul 10 18:30:15.028541 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:15.028123 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-h5p2f" podStartSLOduration=3.028102145 podStartE2EDuration="3.028102145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:14.017103163 +0000 UTC m=+259.222943218" watchObservedRunningTime="2026-07-10 18:30:15.028102145 +0000 UTC m=+260.233942201" Jul 10 18:30:15.028541 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:15.028250 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kq2db" podStartSLOduration=1.742049059 podStartE2EDuration="3.028238793s" totalImagesPullingTime="1.286189734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:15.027774212 +0000 UTC m=+260.233614271" watchObservedRunningTime="2026-07-10 18:30:15.028238793 +0000 UTC m=+260.234078850" Jul 10 18:30:15.113761 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:15.113725 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:15.213554 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:15.213517 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:16.598982 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:16.598952 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:16.599383 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:16.599111 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:16.599383 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:16.599176 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls podName:a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a nodeName:}" failed. No retries permitted until 2026-07-10 18:30:20.599157648 +0000 UTC m=+265.804997700 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dw5tc" (UID: "a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:17.018676 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:17.018639 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" event={"ID":"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d","Type":"ContainerStarted","Data":"64583fce3f56e698d3925f2b60465f6a6e58e7676f85ae613770e1ab38cd6b5f"} Jul 10 18:30:17.020254 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:17.020227 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" event={"ID":"9b56e500-4923-4ab3-8b46-052e2a42839b","Type":"ContainerStarted","Data":"1a226c78fbfcde27f98c88b7b730d36661422baa19ca30530e84eb9e26c7504f"} Jul 10 18:30:17.021722 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:17.021696 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" event={"ID":"aacd5c76-7b3e-4ad4-817c-279d66dae83e","Type":"ContainerStarted","Data":"43785e1fd6e234a3839587493967b90ec3c94dd2b3909210bafa2e4aa25bb66d"} Jul 10 18:30:17.042607 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:17.042551 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" podStartSLOduration=1.826944235 podStartE2EDuration="5.042536441s" totalImagesPullingTime="3.215592206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:17.041970135 +0000 UTC m=+262.247810192" watchObservedRunningTime="2026-07-10 18:30:17.042536441 +0000 UTC m=+262.248376495" Jul 10 18:30:17.061331 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:17.061270 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" podStartSLOduration=1.869079903 podStartE2EDuration="5.06124906s" totalImagesPullingTime="3.192169157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:17.060610493 +0000 UTC m=+262.266450548" watchObservedRunningTime="2026-07-10 18:30:17.06124906 +0000 UTC m=+262.267089116" Jul 10 18:30:17.082095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:17.082046 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" podStartSLOduration=2.003892983 podStartE2EDuration="5.082028208s" totalImagesPullingTime="3.078135225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:17.081377683 +0000 UTC m=+262.287217760" watchObservedRunningTime="2026-07-10 18:30:17.082028208 +0000 UTC m=+262.287868263" Jul 10 18:30:18.086962 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.086928 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:18.182517 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.182459 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:18.286002 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.285916 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:18.286181 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.286156 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9ksnh"] Jul 10 18:30:18.291103 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.291076 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.293944 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.293920 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q6ch7\"" Jul 10 18:30:18.294222 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.294185 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 18:30:18.295174 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.295151 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 18:30:18.300816 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.300791 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9ksnh"] Jul 10 18:30:18.388211 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.388155 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:18.414413 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.414375 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.414555 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.414417 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45wpv\" (UniqueName: \"kubernetes.io/projected/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-api-access-45wpv\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.414555 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.414517 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.414623 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.414601 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d338c47-f51e-4517-98f1-9f4763a838a9-crio-socket\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.414657 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.414631 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d338c47-f51e-4517-98f1-9f4763a838a9-data-volume\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.515715 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.515676 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.515715 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.515719 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wpv\" (UniqueName: \"kubernetes.io/projected/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-api-access-45wpv\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.515942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.515769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.515942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.515806 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d338c47-f51e-4517-98f1-9f4763a838a9-crio-socket\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.515942 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:18.515813 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:30:18.515942 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:18.515899 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm podName:9d338c47-f51e-4517-98f1-9f4763a838a9 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:19.015878336 +0000 UTC m=+264.221718382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9ksnh" (UID: "9d338c47-f51e-4517-98f1-9f4763a838a9") : configmap "kube-rbac-proxy" not found Jul 10 18:30:18.515942 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:18.515916 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:30:18.515942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.515923 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d338c47-f51e-4517-98f1-9f4763a838a9-crio-socket\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.516184 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.515821 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d338c47-f51e-4517-98f1-9f4763a838a9-data-volume\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.516184 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:18.515972 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls podName:9d338c47-f51e-4517-98f1-9f4763a838a9 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:19.015959709 +0000 UTC m=+264.221799754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9ksnh" (UID: "9d338c47-f51e-4517-98f1-9f4763a838a9") : secret "insights-runtime-extractor-tls" not found Jul 10 18:30:18.516184 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.516094 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d338c47-f51e-4517-98f1-9f4763a838a9-data-volume\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.524640 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.524609 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45wpv\" (UniqueName: \"kubernetes.io/projected/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-api-access-45wpv\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:18.901530 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.901494 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:18.985558 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:18.985523 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:19.020692 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:19.020656 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:19.020830 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:19.020749 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:19.020830 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:19.020784 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:30:19.020896 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:19.020863 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm podName:9d338c47-f51e-4517-98f1-9f4763a838a9 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:20.020843467 +0000 UTC m=+265.226683507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9ksnh" (UID: "9d338c47-f51e-4517-98f1-9f4763a838a9") : configmap "kube-rbac-proxy" not found Jul 10 18:30:19.020937 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:19.020907 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:30:19.020989 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:19.020977 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls podName:9d338c47-f51e-4517-98f1-9f4763a838a9 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:20.020962719 +0000 UTC m=+265.226802759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9ksnh" (UID: "9d338c47-f51e-4517-98f1-9f4763a838a9") : secret "insights-runtime-extractor-tls" not found Jul 10 18:30:19.390854 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:19.390797 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:19.445699 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:19.445660 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:19.615572 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:19.615537 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:19.715707 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:19.715612 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:20.029967 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:20.029869 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:20.029967 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:20.029947 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:20.030216 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:20.030030 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:30:20.030216 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:20.030039 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:30:20.030216 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:20.030091 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm podName:9d338c47-f51e-4517-98f1-9f4763a838a9 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:22.03007827 +0000 UTC m=+267.235918304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9ksnh" (UID: "9d338c47-f51e-4517-98f1-9f4763a838a9") : configmap "kube-rbac-proxy" not found Jul 10 18:30:20.030216 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:20.030104 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls podName:9d338c47-f51e-4517-98f1-9f4763a838a9 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:22.030098041 +0000 UTC m=+267.235938073 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9ksnh" (UID: "9d338c47-f51e-4517-98f1-9f4763a838a9") : secret "insights-runtime-extractor-tls" not found Jul 10 18:30:20.177501 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:20.177466 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:20.235677 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:20.235642 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:20.636427 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:20.636381 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:20.636825 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:20.636550 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:20.636825 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:20.636618 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls podName:a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a nodeName:}" failed. No retries permitted until 2026-07-10 18:30:28.636599994 +0000 UTC m=+273.842440026 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dw5tc" (UID: "a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:20.882290 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:20.882246 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7bh5c_f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694/dns-node-resolver/0.log" Jul 10 18:30:21.209169 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.209128 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-nzcgd"] Jul 10 18:30:21.212432 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.212416 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" Jul 10 18:30:21.215510 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.215482 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 10 18:30:21.215794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.215776 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 10 18:30:21.217107 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.217086 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 10 18:30:21.217238 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.217124 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-lhsvh\"" Jul 10 18:30:21.217238 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.217151 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 10 18:30:21.220770 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.220746 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-nzcgd"] Jul 10 18:30:21.245132 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.245087 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:21.342975 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.342943 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7b8b7359-394b-4f1e-a3b1-dd6e176cc164-signing-cabundle\") pod \"service-ca-79f68b9f4f-nzcgd\" (UID: \"7b8b7359-394b-4f1e-a3b1-dd6e176cc164\") " pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" Jul 10 18:30:21.342975 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.342981 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzfk2\" (UniqueName: \"kubernetes.io/projected/7b8b7359-394b-4f1e-a3b1-dd6e176cc164-kube-api-access-zzfk2\") pod \"service-ca-79f68b9f4f-nzcgd\" (UID: \"7b8b7359-394b-4f1e-a3b1-dd6e176cc164\") " pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" Jul 10 18:30:21.343187 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.343031 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7b8b7359-394b-4f1e-a3b1-dd6e176cc164-signing-key\") pod \"service-ca-79f68b9f4f-nzcgd\" (UID: \"7b8b7359-394b-4f1e-a3b1-dd6e176cc164\") " pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" Jul 10 18:30:21.369669 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.369636 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:21.444253 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.444219 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7b8b7359-394b-4f1e-a3b1-dd6e176cc164-signing-cabundle\") pod \"service-ca-79f68b9f4f-nzcgd\" (UID: \"7b8b7359-394b-4f1e-a3b1-dd6e176cc164\") " pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" Jul 10 18:30:21.444253 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.444252 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzfk2\" (UniqueName: \"kubernetes.io/projected/7b8b7359-394b-4f1e-a3b1-dd6e176cc164-kube-api-access-zzfk2\") pod \"service-ca-79f68b9f4f-nzcgd\" (UID: \"7b8b7359-394b-4f1e-a3b1-dd6e176cc164\") " pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" Jul 10 18:30:21.444527 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.444302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7b8b7359-394b-4f1e-a3b1-dd6e176cc164-signing-key\") pod \"service-ca-79f68b9f4f-nzcgd\" (UID: \"7b8b7359-394b-4f1e-a3b1-dd6e176cc164\") " pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" Jul 10 18:30:21.444870 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.444847 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7b8b7359-394b-4f1e-a3b1-dd6e176cc164-signing-cabundle\") pod \"service-ca-79f68b9f4f-nzcgd\" (UID: \"7b8b7359-394b-4f1e-a3b1-dd6e176cc164\") " pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" Jul 10 18:30:21.446709 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.446693 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7b8b7359-394b-4f1e-a3b1-dd6e176cc164-signing-key\") pod \"service-ca-79f68b9f4f-nzcgd\" (UID: \"7b8b7359-394b-4f1e-a3b1-dd6e176cc164\") " pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" Jul 10 18:30:21.452726 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.452707 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzfk2\" (UniqueName: \"kubernetes.io/projected/7b8b7359-394b-4f1e-a3b1-dd6e176cc164-kube-api-access-zzfk2\") pod \"service-ca-79f68b9f4f-nzcgd\" (UID: \"7b8b7359-394b-4f1e-a3b1-dd6e176cc164\") " pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" Jul 10 18:30:21.521845 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.521754 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" Jul 10 18:30:21.644355 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:21.644321 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-nzcgd"] Jul 10 18:30:21.648032 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:21.648006 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b8b7359_394b_4f1e_a3b1_dd6e176cc164.slice/crio-814eb882130a5fe5273b4d6585df2f4247dd35ee5c82c87101816b65c25e2892 WatchSource:0}: Error finding container 814eb882130a5fe5273b4d6585df2f4247dd35ee5c82c87101816b65c25e2892: Status 404 returned error can't find the container with id 814eb882130a5fe5273b4d6585df2f4247dd35ee5c82c87101816b65c25e2892 Jul 10 18:30:22.041108 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:22.041071 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" event={"ID":"7b8b7359-394b-4f1e-a3b1-dd6e176cc164","Type":"ContainerStarted","Data":"b994073c22f6f36dc6487499348e0edc7fd1bced98c160ab1c301783bfaa38e3"} Jul 10 18:30:22.041307 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:22.041121 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" event={"ID":"7b8b7359-394b-4f1e-a3b1-dd6e176cc164","Type":"ContainerStarted","Data":"814eb882130a5fe5273b4d6585df2f4247dd35ee5c82c87101816b65c25e2892"} Jul 10 18:30:22.049633 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:22.049610 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:22.049777 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:22.049709 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:22.049777 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:22.049743 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:30:22.049854 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:22.049801 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls podName:9d338c47-f51e-4517-98f1-9f4763a838a9 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:26.049783757 +0000 UTC m=+271.255623794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9ksnh" (UID: "9d338c47-f51e-4517-98f1-9f4763a838a9") : secret "insights-runtime-extractor-tls" not found Jul 10 18:30:22.049854 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:22.049814 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:30:22.049934 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:22.049856 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm podName:9d338c47-f51e-4517-98f1-9f4763a838a9 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:26.04984463 +0000 UTC m=+271.255684663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9ksnh" (UID: "9d338c47-f51e-4517-98f1-9f4763a838a9") : configmap "kube-rbac-proxy" not found Jul 10 18:30:22.060915 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:22.060868 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-nzcgd" podStartSLOduration=1.060853867 podStartE2EDuration="1.060853867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:22.059735372 +0000 UTC m=+267.265575448" watchObservedRunningTime="2026-07-10 18:30:22.060853867 +0000 UTC m=+267.266693922" Jul 10 18:30:22.134017 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:22.133981 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:22.283014 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:22.282981 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qjh75_001a040d-8e63-4eac-8af4-5ce45a0e4914/node-ca/0.log" Jul 10 18:30:22.683801 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:22.683703 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-psq6h_c5ff392a-52f5-4430-9b45-c07ecc3445ec/serve-healthcheck-canary/0.log" Jul 10 18:30:22.722576 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:22.722542 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:22.822566 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:22.822529 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:23.017406 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:23.017320 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:23.131668 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:23.131634 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:23.223814 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:23.223778 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:23.425128 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:23.425090 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:23.685060 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:23.684975 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-6kgms_9b56e500-4923-4ab3-8b46-052e2a42839b/kube-storage-version-migrator-operator/0.log" Jul 10 18:30:24.217609 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:24.217570 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:24.424053 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:24.424018 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:25.044260 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:25.044222 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:25.380286 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:25.380255 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:25.477254 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:25.477215 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:25.826637 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:25.826546 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:26.026344 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:26.026307 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:26.086360 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:26.086268 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:26.086741 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:26.086359 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:30:26.086741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:26.086372 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:26.086741 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:26.086421 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm podName:9d338c47-f51e-4517-98f1-9f4763a838a9 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:34.086402893 +0000 UTC m=+279.292242951 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9ksnh" (UID: "9d338c47-f51e-4517-98f1-9f4763a838a9") : configmap "kube-rbac-proxy" not found Jul 10 18:30:26.086741 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:26.086472 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:30:26.086741 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:26.086525 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls podName:9d338c47-f51e-4517-98f1-9f4763a838a9 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:34.086515896 +0000 UTC m=+279.292355928 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9ksnh" (UID: "9d338c47-f51e-4517-98f1-9f4763a838a9") : secret "insights-runtime-extractor-tls" not found Jul 10 18:30:27.230921 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:27.230881 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:27.331504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:27.331466 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:27.617500 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:27.617472 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:28.620099 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:28.620067 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:28.672347 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:28.672312 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:28.710857 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:28.710817 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:28.711009 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:28.710959 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:28.711054 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:28.711018 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls podName:a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a nodeName:}" failed. No retries permitted until 2026-07-10 18:30:44.71100121 +0000 UTC m=+289.916841243 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dw5tc" (UID: "a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:30:29.927274 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:29.927244 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:30.029246 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:30.029217 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:30.095291 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:30.095252 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:30.217292 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:30.217164 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:31.215001 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:31.214964 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:31.316563 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:31.316529 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:31.612047 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:31.612013 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:31.810161 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:31.810121 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:32.013738 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:32.013641 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:32.412269 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:32.412234 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:32.815752 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:32.815665 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:34.027547 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:34.027513 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:34.096724 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:34.096690 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:34.159931 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:34.159896 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:34.160079 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:34.159982 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:34.160463 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:34.160442 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d338c47-f51e-4517-98f1-9f4763a838a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:34.162249 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:34.162232 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d338c47-f51e-4517-98f1-9f4763a838a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9ksnh\" (UID: \"9d338c47-f51e-4517-98f1-9f4763a838a9\") " pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:34.201454 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:34.201418 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9ksnh" Jul 10 18:30:34.345987 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:34.345914 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9ksnh"] Jul 10 18:30:34.348456 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:34.348423 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d338c47_f51e_4517_98f1_9f4763a838a9.slice/crio-62cb2fc7759b7db62b4deb65b751d470c7a2a19f7b2d69d420e525c5613baeb7 WatchSource:0}: Error finding container 62cb2fc7759b7db62b4deb65b751d470c7a2a19f7b2d69d420e525c5613baeb7: Status 404 returned error can't find the container with id 62cb2fc7759b7db62b4deb65b751d470c7a2a19f7b2d69d420e525c5613baeb7 Jul 10 18:30:35.084388 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:35.084350 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9ksnh" event={"ID":"9d338c47-f51e-4517-98f1-9f4763a838a9","Type":"ContainerStarted","Data":"62cb2fc7759b7db62b4deb65b751d470c7a2a19f7b2d69d420e525c5613baeb7"} Jul 10 18:30:35.748513 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:35.748463 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" podUID="5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac" Jul 10 18:30:35.748513 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:35.748462 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" podUID="2f08196c-7a64-43e1-9dd4-bf32a34b2844" Jul 10 18:30:35.748762 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:35.748457 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5ft29" podUID="06300a4b-433b-4fa0-b969-78d216179a3b" Jul 10 18:30:36.088032 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:36.087955 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5ft29" Jul 10 18:30:36.088032 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:36.087978 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:30:36.088525 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:36.088257 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:30:38.095356 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:38.095328 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9ksnh" event={"ID":"9d338c47-f51e-4517-98f1-9f4763a838a9","Type":"ContainerStarted","Data":"aa8af36347eb5c6efa4564f15a9d2053792e8bb44a3262ba1cdff41f6a931913"} Jul 10 18:30:39.105907 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.105822 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9ksnh" event={"ID":"9d338c47-f51e-4517-98f1-9f4763a838a9","Type":"ContainerStarted","Data":"67876a2aa59464de988135917c135251e9db5dd072c878f0dec76959d9bbdd42"} Jul 10 18:30:39.477164 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.477128 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:39.598152 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.598083 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:39.709904 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.709871 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:30:39.710078 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.709931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:30:39.710078 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.709965 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:30:39.712604 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.712580 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/06300a4b-433b-4fa0-b969-78d216179a3b-metrics-tls\") pod \"dns-default-5ft29\" (UID: \"06300a4b-433b-4fa0-b969-78d216179a3b\") " pod="openshift-dns/dns-default-5ft29" Jul 10 18:30:39.712746 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.712723 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vt5c2\" (UID: \"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:30:39.712788 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.712744 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"image-registry-67b79b8bcc-fwqbw\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:30:39.975721 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.975665 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:39.993262 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.993230 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-lkrwm\"" Jul 10 18:30:39.993262 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.993230 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rc4ck\"" Jul 10 18:30:39.993503 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.993270 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-z2lbt\"" Jul 10 18:30:39.999356 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.999330 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5ft29" Jul 10 18:30:39.999356 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.999366 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" Jul 10 18:30:39.999559 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:39.999344 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:30:40.165433 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:40.165340 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:40.186447 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:40.186203 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vt5c2"] Jul 10 18:30:40.189919 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:40.189888 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bc9f8f6_2ed4_4cce_ac5c_2f4f1ce093ac.slice/crio-cd50f2c4b0c82da77752624857b505aca882f720ff75629b838b80fc6d28e721 WatchSource:0}: Error finding container cd50f2c4b0c82da77752624857b505aca882f720ff75629b838b80fc6d28e721: Status 404 returned error can't find the container with id cd50f2c4b0c82da77752624857b505aca882f720ff75629b838b80fc6d28e721 Jul 10 18:30:40.266079 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:40.265982 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:40.405598 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:40.405567 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:40.419451 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:40.419340 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5ft29"] Jul 10 18:30:40.424281 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:40.424253 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67b79b8bcc-fwqbw"] Jul 10 18:30:40.466495 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:40.466457 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:41.117962 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:41.117927 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" event={"ID":"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac","Type":"ContainerStarted","Data":"cd50f2c4b0c82da77752624857b505aca882f720ff75629b838b80fc6d28e721"} Jul 10 18:30:41.367527 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:41.367488 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:41.391611 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:41.391544 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06300a4b_433b_4fa0_b969_78d216179a3b.slice/crio-79cbffcfbae02c4e98c4bbf3d03a5beb06ce19911ecf7f6345b41bc75fde82e4 WatchSource:0}: Error finding container 79cbffcfbae02c4e98c4bbf3d03a5beb06ce19911ecf7f6345b41bc75fde82e4: Status 404 returned error can't find the container with id 79cbffcfbae02c4e98c4bbf3d03a5beb06ce19911ecf7f6345b41bc75fde82e4 Jul 10 18:30:41.392589 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:41.392448 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f08196c_7a64_43e1_9dd4_bf32a34b2844.slice/crio-ff1174d051dee9a6179b71a835377782afaed28516efcaca32e26af6a7f314a6 WatchSource:0}: Error finding container ff1174d051dee9a6179b71a835377782afaed28516efcaca32e26af6a7f314a6: Status 404 returned error can't find the container with id ff1174d051dee9a6179b71a835377782afaed28516efcaca32e26af6a7f314a6 Jul 10 18:30:41.467571 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:41.467535 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:42.124603 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:42.124564 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9ksnh" event={"ID":"9d338c47-f51e-4517-98f1-9f4763a838a9","Type":"ContainerStarted","Data":"aa91b10ce49cae799e9eceaafec7d8c076b9e55e4269f91aa19ec1addcdd412a"} Jul 10 18:30:42.126340 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:42.126301 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5ft29" event={"ID":"06300a4b-433b-4fa0-b969-78d216179a3b","Type":"ContainerStarted","Data":"79cbffcfbae02c4e98c4bbf3d03a5beb06ce19911ecf7f6345b41bc75fde82e4"} Jul 10 18:30:42.128147 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:42.128112 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" event={"ID":"2f08196c-7a64-43e1-9dd4-bf32a34b2844","Type":"ContainerStarted","Data":"39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f"} Jul 10 18:30:42.128327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:42.128154 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" event={"ID":"2f08196c-7a64-43e1-9dd4-bf32a34b2844","Type":"ContainerStarted","Data":"ff1174d051dee9a6179b71a835377782afaed28516efcaca32e26af6a7f314a6"} Jul 10 18:30:42.128327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:42.128248 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:30:42.149807 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:42.149740 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9ksnh" podStartSLOduration=17.169502521 podStartE2EDuration="24.149716766s" totalImagesPullingTime="6.980214245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:30:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:42.148572675 +0000 UTC m=+287.354412755" watchObservedRunningTime="2026-07-10 18:30:42.149716766 +0000 UTC m=+287.355556824" Jul 10 18:30:42.173778 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:42.173710 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" podStartSLOduration=264.173686594 podStartE2EDuration="4m24.173686594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:26:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:42.172607966 +0000 UTC m=+287.378448045" watchObservedRunningTime="2026-07-10 18:30:42.173686594 +0000 UTC m=+287.379526629" Jul 10 18:30:42.686930 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:42.686890 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:43.133818 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:43.133786 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" event={"ID":"5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac","Type":"ContainerStarted","Data":"81871d3945c00b6883e376314879cae046aff9f947b7af257b4b59c1f2e371b6"} Jul 10 18:30:43.158034 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:43.157976 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-vt5c2" podStartSLOduration=266.067089057 podStartE2EDuration="4m28.157954155s" totalImagesPullingTime="2.090865098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:43.155764811 +0000 UTC m=+288.361604895" watchObservedRunningTime="2026-07-10 18:30:43.157954155 +0000 UTC m=+288.363794189" Jul 10 18:30:44.138548 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.138510 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5ft29" event={"ID":"06300a4b-433b-4fa0-b969-78d216179a3b","Type":"ContainerStarted","Data":"bc2492f1e7961aea404cf2fa535def1cfc38f2b92d217f4458366995061bb581"} Jul 10 18:30:44.138990 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.138573 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5ft29" event={"ID":"06300a4b-433b-4fa0-b969-78d216179a3b","Type":"ContainerStarted","Data":"95a6b0c62ac625fef0a34b3ba323ab1ebd2fc13f11e8bff47e1c44ea5825f601"} Jul 10 18:30:44.169978 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.169929 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5ft29" podStartSLOduration=253.276553722 podStartE2EDuration="4m15.169914511s" totalImagesPullingTime="1.893360789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:44.169530489 +0000 UTC m=+289.375370546" watchObservedRunningTime="2026-07-10 18:30:44.169914511 +0000 UTC m=+289.375754566" Jul 10 18:30:44.182808 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.182780 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-hsvxt"] Jul 10 18:30:44.186015 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.185997 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-hsvxt" Jul 10 18:30:44.194053 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.194033 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hz2bt\"" Jul 10 18:30:44.194623 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.194608 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 18:30:44.195152 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.195132 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 18:30:44.211873 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.211849 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-hsvxt"] Jul 10 18:30:44.251076 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.251034 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfcvb\" (UniqueName: \"kubernetes.io/projected/9c12efa8-f9e8-46a2-8ad3-3fa3e645dcde-kube-api-access-qfcvb\") pod \"downloads-68c9cbb497-hsvxt\" (UID: \"9c12efa8-f9e8-46a2-8ad3-3fa3e645dcde\") " pod="openshift-console/downloads-68c9cbb497-hsvxt" Jul 10 18:30:44.312322 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.312285 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:44.352494 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.352461 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfcvb\" (UniqueName: \"kubernetes.io/projected/9c12efa8-f9e8-46a2-8ad3-3fa3e645dcde-kube-api-access-qfcvb\") pod \"downloads-68c9cbb497-hsvxt\" (UID: \"9c12efa8-f9e8-46a2-8ad3-3fa3e645dcde\") " pod="openshift-console/downloads-68c9cbb497-hsvxt" Jul 10 18:30:44.361478 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.361446 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfcvb\" (UniqueName: \"kubernetes.io/projected/9c12efa8-f9e8-46a2-8ad3-3fa3e645dcde-kube-api-access-qfcvb\") pod \"downloads-68c9cbb497-hsvxt\" (UID: \"9c12efa8-f9e8-46a2-8ad3-3fa3e645dcde\") " pod="openshift-console/downloads-68c9cbb497-hsvxt" Jul 10 18:30:44.395614 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.395529 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:44.495795 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.495754 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-hsvxt" Jul 10 18:30:44.500325 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.500294 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:44.665901 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.665873 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-hsvxt"] Jul 10 18:30:44.666913 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:44.666888 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c12efa8_f9e8_46a2_8ad3_3fa3e645dcde.slice/crio-b792c4c91da32642590f60e496ceca5545b410f8923f9ac58345eb1dade438e7 WatchSource:0}: Error finding container b792c4c91da32642590f60e496ceca5545b410f8923f9ac58345eb1dade438e7: Status 404 returned error can't find the container with id b792c4c91da32642590f60e496ceca5545b410f8923f9ac58345eb1dade438e7 Jul 10 18:30:44.672712 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.672685 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:44.756516 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.756480 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:44.758851 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.758822 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dw5tc\" (UID: \"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:44.773562 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.773532 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:44.987209 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:44.987098 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" Jul 10 18:30:45.124704 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:45.124625 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc"] Jul 10 18:30:45.127505 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:45.127476 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda83a3d6f_a993_4fc5_ac83_a8b2ab4b028a.slice/crio-1b842e3a948e7ddd26c4a2d0dd15cff6ea645751df9eccf3e46ddc069c098f37 WatchSource:0}: Error finding container 1b842e3a948e7ddd26c4a2d0dd15cff6ea645751df9eccf3e46ddc069c098f37: Status 404 returned error can't find the container with id 1b842e3a948e7ddd26c4a2d0dd15cff6ea645751df9eccf3e46ddc069c098f37 Jul 10 18:30:45.144175 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:45.144112 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-hsvxt" event={"ID":"9c12efa8-f9e8-46a2-8ad3-3fa3e645dcde","Type":"ContainerStarted","Data":"b792c4c91da32642590f60e496ceca5545b410f8923f9ac58345eb1dade438e7"} Jul 10 18:30:45.145455 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:45.145428 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" event={"ID":"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a","Type":"ContainerStarted","Data":"1b842e3a948e7ddd26c4a2d0dd15cff6ea645751df9eccf3e46ddc069c098f37"} Jul 10 18:30:45.145604 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:45.145583 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5ft29" Jul 10 18:30:47.156642 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.156605 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" event={"ID":"a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a","Type":"ContainerStarted","Data":"b5b7de020f24c246b8164f53fefd4dd0bc41d764b7c0de375e4bfdf173d48616"} Jul 10 18:30:47.188310 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.188257 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:47.259637 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.259553 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:47.342721 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.342655 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dw5tc" podStartSLOduration=33.630532828 podStartE2EDuration="35.342630434s" totalImagesPullingTime="1.712097606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:47.177707011 +0000 UTC m=+292.383547081" watchObservedRunningTime="2026-07-10 18:30:47.342630434 +0000 UTC m=+292.548470489" Jul 10 18:30:47.342988 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.342970 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962"] Jul 10 18:30:47.346328 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.346302 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962" Jul 10 18:30:47.349331 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.349075 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 10 18:30:47.349331 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.349076 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-p42w2\"" Jul 10 18:30:47.357496 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.357431 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962"] Jul 10 18:30:47.376330 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.376297 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:47.382208 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.382166 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7147b18c-c060-410f-b82f-27beaeea9f60-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-vg962\" (UID: \"7147b18c-c060-410f-b82f-27beaeea9f60\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962" Jul 10 18:30:47.483707 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.483624 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7147b18c-c060-410f-b82f-27beaeea9f60-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-vg962\" (UID: \"7147b18c-c060-410f-b82f-27beaeea9f60\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962" Jul 10 18:30:47.483865 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:47.483794 2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 10 18:30:47.483921 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:47.483874 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7147b18c-c060-410f-b82f-27beaeea9f60-tls-certificates podName:7147b18c-c060-410f-b82f-27beaeea9f60 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:47.983852809 +0000 UTC m=+293.189692849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/7147b18c-c060-410f-b82f-27beaeea9f60-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-vg962" (UID: "7147b18c-c060-410f-b82f-27beaeea9f60") : secret "prometheus-operator-admission-webhook-tls" not found Jul 10 18:30:47.596607 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.596571 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-568dc68b94-sstf6"] Jul 10 18:30:47.599999 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.599968 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.603384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.603258 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 18:30:47.603521 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.603407 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 18:30:47.603635 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.603617 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-grfkv\"" Jul 10 18:30:47.603704 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.603644 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 18:30:47.603886 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.603863 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 18:30:47.604119 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.603935 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 18:30:47.611783 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.611657 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-568dc68b94-sstf6"] Jul 10 18:30:47.684790 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.684757 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-oauth-serving-cert\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.684983 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.684807 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-oauth-config\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.684983 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.684837 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bxwl\" (UniqueName: \"kubernetes.io/projected/fcd36080-d958-459c-96d2-8157766352cd-kube-api-access-5bxwl\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.684983 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.684910 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-service-ca\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.684983 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.684942 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-serving-cert\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.685178 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.685048 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-console-config\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.785820 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.785703 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-oauth-serving-cert\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.785820 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.785760 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-oauth-config\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.785820 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.785789 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bxwl\" (UniqueName: \"kubernetes.io/projected/fcd36080-d958-459c-96d2-8157766352cd-kube-api-access-5bxwl\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.785820 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.785816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-service-ca\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.786136 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.785845 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-serving-cert\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.786616 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.786497 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-console-config\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.786757 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.786635 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-oauth-serving-cert\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.786757 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.786671 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-service-ca\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.786964 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.786943 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-console-config\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.788885 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.788859 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-serving-cert\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.789277 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.789259 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-oauth-config\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.794976 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.794956 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bxwl\" (UniqueName: \"kubernetes.io/projected/fcd36080-d958-459c-96d2-8157766352cd-kube-api-access-5bxwl\") pod \"console-568dc68b94-sstf6\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.912873 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.912842 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:47.920602 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.920547 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:47.989434 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.989399 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7147b18c-c060-410f-b82f-27beaeea9f60-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-vg962\" (UID: \"7147b18c-c060-410f-b82f-27beaeea9f60\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962" Jul 10 18:30:47.992521 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:47.992437 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7147b18c-c060-410f-b82f-27beaeea9f60-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-vg962\" (UID: \"7147b18c-c060-410f-b82f-27beaeea9f60\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962" Jul 10 18:30:48.071188 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:48.071013 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-568dc68b94-sstf6"] Jul 10 18:30:48.074127 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:48.074099 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd36080_d958_459c_96d2_8157766352cd.slice/crio-74da8aeefcf76e87405b1ccc37219c34ac9cbb2ccd8397f88700e029d922374b WatchSource:0}: Error finding container 74da8aeefcf76e87405b1ccc37219c34ac9cbb2ccd8397f88700e029d922374b: Status 404 returned error can't find the container with id 74da8aeefcf76e87405b1ccc37219c34ac9cbb2ccd8397f88700e029d922374b Jul 10 18:30:48.161521 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:48.161471 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568dc68b94-sstf6" event={"ID":"fcd36080-d958-459c-96d2-8157766352cd","Type":"ContainerStarted","Data":"74da8aeefcf76e87405b1ccc37219c34ac9cbb2ccd8397f88700e029d922374b"} Jul 10 18:30:48.260276 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:48.260234 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962" Jul 10 18:30:48.412560 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:48.412529 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962"] Jul 10 18:30:48.415600 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:48.415575 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7147b18c_c060_410f_b82f_27beaeea9f60.slice/crio-fae3411882669c83ac52973b5b8f5f1d099e2b24ef2c1fd59405553d5ce64b8d WatchSource:0}: Error finding container fae3411882669c83ac52973b5b8f5f1d099e2b24ef2c1fd59405553d5ce64b8d: Status 404 returned error can't find the container with id fae3411882669c83ac52973b5b8f5f1d099e2b24ef2c1fd59405553d5ce64b8d Jul 10 18:30:49.167914 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:49.167842 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962" event={"ID":"7147b18c-c060-410f-b82f-27beaeea9f60","Type":"ContainerStarted","Data":"fae3411882669c83ac52973b5b8f5f1d099e2b24ef2c1fd59405553d5ce64b8d"} Jul 10 18:30:50.174409 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.174376 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962" event={"ID":"7147b18c-c060-410f-b82f-27beaeea9f60","Type":"ContainerStarted","Data":"72a76594ab241e840fbd291baadee859ced4f02c6585a0b1b8140f4c1adc2d79"} Jul 10 18:30:50.174841 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.174576 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962" Jul 10 18:30:50.179796 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.179768 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962" Jul 10 18:30:50.191535 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.191467 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-vg962" podStartSLOduration=1.977448589 podStartE2EDuration="3.19144953s" totalImagesPullingTime="1.214000941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:50.190381621 +0000 UTC m=+295.396221678" watchObservedRunningTime="2026-07-10 18:30:50.19144953 +0000 UTC m=+295.397289616" Jul 10 18:30:50.372454 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.372372 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:50.407459 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.407370 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rz96w"] Jul 10 18:30:50.426427 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.425617 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rz96w"] Jul 10 18:30:50.426427 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.425738 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:50.429157 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.429078 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 10 18:30:50.429837 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.429576 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jg42w\"" Jul 10 18:30:50.429837 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.429649 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 10 18:30:50.430051 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.429880 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 18:30:50.446655 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.446622 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:50.513174 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.513135 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:50.513382 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.513239 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:50.513382 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.513274 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxl7r\" (UniqueName: \"kubernetes.io/projected/603dd2d0-e930-4d54-83ad-090a381b2846-kube-api-access-wxl7r\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:50.513382 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.513307 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/603dd2d0-e930-4d54-83ad-090a381b2846-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:50.614728 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.614690 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxl7r\" (UniqueName: \"kubernetes.io/projected/603dd2d0-e930-4d54-83ad-090a381b2846-kube-api-access-wxl7r\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:50.614919 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.614745 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/603dd2d0-e930-4d54-83ad-090a381b2846-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:50.614919 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.614796 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:50.614919 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.614864 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:50.615087 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:50.614965 2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 10 18:30:50.615087 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:50.615030 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-tls podName:603dd2d0-e930-4d54-83ad-090a381b2846 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:51.115010567 +0000 UTC m=+296.320850616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-tls") pod "prometheus-operator-64785b6675-rz96w" (UID: "603dd2d0-e930-4d54-83ad-090a381b2846") : secret "prometheus-operator-tls" not found Jul 10 18:30:50.615790 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.615765 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/603dd2d0-e930-4d54-83ad-090a381b2846-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:50.618101 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.618070 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:50.626309 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:50.626258 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxl7r\" (UniqueName: \"kubernetes.io/projected/603dd2d0-e930-4d54-83ad-090a381b2846-kube-api-access-wxl7r\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:51.120568 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:51.120526 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:51.120768 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:51.120719 2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 10 18:30:51.120841 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:51.120787 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-tls podName:603dd2d0-e930-4d54-83ad-090a381b2846 nodeName:}" failed. No retries permitted until 2026-07-10 18:30:52.120769601 +0000 UTC m=+297.326609634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-tls") pod "prometheus-operator-64785b6675-rz96w" (UID: "603dd2d0-e930-4d54-83ad-090a381b2846") : secret "prometheus-operator-tls" not found Jul 10 18:30:51.314935 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:51.314890 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:51.414872 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:51.414782 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:52.130005 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:52.129969 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:52.132269 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:52.132241 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/603dd2d0-e930-4d54-83ad-090a381b2846-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rz96w\" (UID: \"603dd2d0-e930-4d54-83ad-090a381b2846\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:52.187736 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:52.187702 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568dc68b94-sstf6" event={"ID":"fcd36080-d958-459c-96d2-8157766352cd","Type":"ContainerStarted","Data":"c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658"} Jul 10 18:30:52.205647 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:52.205592 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-568dc68b94-sstf6" podStartSLOduration=1.7139401319999998 podStartE2EDuration="5.205575873s" totalImagesPullingTime="3.491635741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:52.204507825 +0000 UTC m=+297.410347917" watchObservedRunningTime="2026-07-10 18:30:52.205575873 +0000 UTC m=+297.411415928" Jul 10 18:30:52.237727 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:52.237692 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" Jul 10 18:30:52.290241 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:52.290185 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-tvbm5" podUID="24908795-9d74-4fec-bb0b-859f3b202d44" Jul 10 18:30:52.380573 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:52.380550 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rz96w"] Jul 10 18:30:52.383171 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:30:52.383144 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603dd2d0_e930_4d54_83ad_090a381b2846.slice/crio-11112208fd90e8dfa07dea2312ab3014e547c6dd135ef29d1a6d5a5bff820405 WatchSource:0}: Error finding container 11112208fd90e8dfa07dea2312ab3014e547c6dd135ef29d1a6d5a5bff820405: Status 404 returned error can't find the container with id 11112208fd90e8dfa07dea2312ab3014e547c6dd135ef29d1a6d5a5bff820405 Jul 10 18:30:53.194228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:53.194176 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" event={"ID":"603dd2d0-e930-4d54-83ad-090a381b2846","Type":"ContainerStarted","Data":"11112208fd90e8dfa07dea2312ab3014e547c6dd135ef29d1a6d5a5bff820405"} Jul 10 18:30:53.586287 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:53.586181 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:53.639649 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:53.639601 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:54.010095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:54.010050 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:54.200829 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:54.200796 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" event={"ID":"603dd2d0-e930-4d54-83ad-090a381b2846","Type":"ContainerStarted","Data":"1631af5ff667a990a078125b2cf9474003704c4da99d73c627d1a94c407acb4c"} Jul 10 18:30:54.201021 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:54.200848 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" event={"ID":"603dd2d0-e930-4d54-83ad-090a381b2846","Type":"ContainerStarted","Data":"26c340e112a16601fed8eee41e396081cb032cc0713fb21392f0cf4ebb58da6d"} Jul 10 18:30:54.222437 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:54.222369 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-rz96w" podStartSLOduration=2.88184633 podStartE2EDuration="4.222347189s" totalImagesPullingTime="1.340500859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:54.220523711 +0000 UTC m=+299.426363790" watchObservedRunningTime="2026-07-10 18:30:54.222347189 +0000 UTC m=+299.428187246" Jul 10 18:30:55.152314 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.152284 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5ft29" Jul 10 18:30:55.160207 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.160159 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:30:55.164008 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.163985 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:30:55.166493 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.166472 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 18:30:55.509687 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.509592 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:55.629097 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.629056 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:55.728951 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.728917 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:55.792241 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.792150 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ltvfr"] Jul 10 18:30:55.795820 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.795792 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.799096 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.798872 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 18:30:55.799096 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.798904 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vg69v\"" Jul 10 18:30:55.799305 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.799184 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 18:30:55.799413 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.799392 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 18:30:55.826073 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.826024 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:55.865465 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.865432 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4d9c257e-3e4b-4017-a462-6c37c213eddd-root\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.865635 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.865485 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-wtmp\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.865635 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.865515 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.865635 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.865545 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmcvd\" (UniqueName: \"kubernetes.io/projected/4d9c257e-3e4b-4017-a462-6c37c213eddd-kube-api-access-vmcvd\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.865635 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.865571 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-textfile\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.865635 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.865599 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d9c257e-3e4b-4017-a462-6c37c213eddd-sys\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.865907 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.865636 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-accelerators-collector-config\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.865907 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.865669 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-tls\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.865907 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.865692 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d9c257e-3e4b-4017-a462-6c37c213eddd-metrics-client-ca\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.899892 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.899861 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:55.967214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.966958 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4d9c257e-3e4b-4017-a462-6c37c213eddd-root\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967008 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-wtmp\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967038 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967072 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmcvd\" (UniqueName: \"kubernetes.io/projected/4d9c257e-3e4b-4017-a462-6c37c213eddd-kube-api-access-vmcvd\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967095 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-textfile\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967100 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4d9c257e-3e4b-4017-a462-6c37c213eddd-root\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967120 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d9c257e-3e4b-4017-a462-6c37c213eddd-sys\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967167 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d9c257e-3e4b-4017-a462-6c37c213eddd-sys\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967167 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-wtmp\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-accelerators-collector-config\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967839 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967248 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-tls\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967839 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967273 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d9c257e-3e4b-4017-a462-6c37c213eddd-metrics-client-ca\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.967839 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:55.967469 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 18:30:55.967839 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:30:55.967548 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-tls podName:4d9c257e-3e4b-4017-a462-6c37c213eddd nodeName:}" failed. No retries permitted until 2026-07-10 18:30:56.467524397 +0000 UTC m=+301.673364452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-tls") pod "node-exporter-ltvfr" (UID: "4d9c257e-3e4b-4017-a462-6c37c213eddd") : secret "node-exporter-tls" not found Jul 10 18:30:55.967839 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.967552 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-textfile\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.968213 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.968176 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d9c257e-3e4b-4017-a462-6c37c213eddd-metrics-client-ca\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.968294 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.968176 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-accelerators-collector-config\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.970669 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.970632 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:55.988228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:55.988068 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmcvd\" (UniqueName: \"kubernetes.io/projected/4d9c257e-3e4b-4017-a462-6c37c213eddd-kube-api-access-vmcvd\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:56.022321 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.022286 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:56.122932 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.122887 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:56.199793 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.199745 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:56.452928 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.452842 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:56.471937 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.471901 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-tls\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:56.474760 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.474729 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4d9c257e-3e4b-4017-a462-6c37c213eddd-node-exporter-tls\") pod \"node-exporter-ltvfr\" (UID: \"4d9c257e-3e4b-4017-a462-6c37c213eddd\") " pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:56.709914 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.708474 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ltvfr" Jul 10 18:30:56.743838 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.743630 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:56.832880 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.832829 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:56.835562 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.835524 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:30:56.840087 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.840060 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.843518 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.843489 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 18:30:56.843518 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.843509 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 18:30:56.843762 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.843533 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 18:30:56.843762 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.843612 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-79ndf\"" Jul 10 18:30:56.843762 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.843493 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 18:30:56.844053 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.844033 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 18:30:56.844154 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.844074 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 18:30:56.844724 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.844556 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 18:30:56.844724 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.844556 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 18:30:56.850493 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.850290 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 18:30:56.855500 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.855472 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:30:56.952316 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.952276 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:56.977385 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977300 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-tls-assets\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977550 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977407 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977550 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977444 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-volume\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977550 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977475 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977693 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977543 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-web-config\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977693 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977577 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlkl8\" (UniqueName: \"kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-kube-api-access-jlkl8\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977693 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977659 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-out\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977816 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977694 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977816 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977737 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977816 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977767 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977816 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977804 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977977 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977870 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:56.977977 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:56.977903 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.050358 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.050317 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:57.078565 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.078526 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-web-config\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.078733 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.078670 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlkl8\" (UniqueName: \"kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-kube-api-access-jlkl8\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.078813 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.078743 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-out\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.078813 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.078769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.078895 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.078811 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.078895 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.078844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.078895 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.078878 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.078996 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.078914 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.078996 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.078934 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.079107 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.079009 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-tls-assets\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.079107 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.079073 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.079107 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.079098 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-volume\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.079277 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.079121 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.079572 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.079350 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.080177 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.080154 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.080976 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.080946 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.082664 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.082629 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-web-config\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.083167 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.083143 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.083752 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.083727 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.083991 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.083949 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-out\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.084227 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.084183 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.084323 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.084254 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.084695 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.084673 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-volume\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.084864 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.084848 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-tls-assets\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.085317 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.085299 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.088162 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.088142 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlkl8\" (UniqueName: \"kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-kube-api-access-jlkl8\") pod \"alertmanager-main-0\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.155329 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.155291 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:30:57.393739 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.393702 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:57.502885 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.502846 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:57.913896 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.913862 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:57.914104 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.913927 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:57.919884 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:57.919855 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:58.227572 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.227492 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:30:58.729596 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.729542 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:58.830230 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.830137 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:58.854973 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.852960 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6"] Jul 10 18:30:58.857317 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.857292 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:58.860723 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.860292 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 10 18:30:58.860723 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.860341 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-b64uteuikemvu\"" Jul 10 18:30:58.860723 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.860358 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 10 18:30:58.860723 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.860492 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 10 18:30:58.861017 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.860822 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 10 18:30:58.861017 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.860869 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 10 18:30:58.861017 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.860922 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-jfbjx\"" Jul 10 18:30:58.872639 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.872615 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6"] Jul 10 18:30:58.996135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.996053 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:58.996135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.996111 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:58.996366 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.996170 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htx7x\" (UniqueName: \"kubernetes.io/projected/9493119a-77ed-47bf-b4bc-2e1315e24e9b-kube-api-access-htx7x\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:58.996366 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.996226 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9493119a-77ed-47bf-b4bc-2e1315e24e9b-metrics-client-ca\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:58.996366 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.996262 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:58.996366 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.996292 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:58.996366 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.996325 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-tls\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:58.996621 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:58.996379 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-grpc-tls\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.007163 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.007121 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:59.097123 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.097084 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.097329 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.097145 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.097329 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.097186 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htx7x\" (UniqueName: \"kubernetes.io/projected/9493119a-77ed-47bf-b4bc-2e1315e24e9b-kube-api-access-htx7x\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.097329 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.097243 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9493119a-77ed-47bf-b4bc-2e1315e24e9b-metrics-client-ca\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.097329 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.097285 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.097329 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.097312 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.097605 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.097343 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-tls\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.097605 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.097397 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-grpc-tls\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.098364 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.098311 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9493119a-77ed-47bf-b4bc-2e1315e24e9b-metrics-client-ca\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.100560 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.100527 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-grpc-tls\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.100664 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.100615 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.101080 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.101053 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.101204 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.101171 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-tls\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.101694 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.101669 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.102544 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.102519 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9493119a-77ed-47bf-b4bc-2e1315e24e9b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.106787 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.106763 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htx7x\" (UniqueName: \"kubernetes.io/projected/9493119a-77ed-47bf-b4bc-2e1315e24e9b-kube-api-access-htx7x\") pod \"thanos-querier-7bf7c7f8f7-rlpl6\" (UID: \"9493119a-77ed-47bf-b4bc-2e1315e24e9b\") " pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.169920 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.169875 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:30:59.301776 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:30:59.301682 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:00.005330 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.005294 2565 patch_prober.go:28] interesting pod/image-registry-67b79b8bcc-fwqbw container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 18:31:00.005781 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.005359 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" podUID="2f08196c-7a64-43e1-9dd4-bf32a34b2844" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 18:31:00.068872 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.068839 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-c89ffd77f-v4q2w"] Jul 10 18:31:00.072399 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.072374 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.076910 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.076883 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 10 18:31:00.077052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.076883 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-hf7j9\"" Jul 10 18:31:00.077052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.076928 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 10 18:31:00.077052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.076992 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bgvdm67lfd1fj\"" Jul 10 18:31:00.077052 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.076887 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 10 18:31:00.077280 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.077272 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 18:31:00.085543 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.085515 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c89ffd77f-v4q2w"] Jul 10 18:31:00.207888 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.207845 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hdjr\" (UniqueName: \"kubernetes.io/projected/a3198dde-3bee-4659-b217-35286c1c7611-kube-api-access-7hdjr\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.208060 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.207906 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a3198dde-3bee-4659-b217-35286c1c7611-secret-metrics-server-client-certs\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.208060 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.207985 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a3198dde-3bee-4659-b217-35286c1c7611-audit-log\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.208060 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.208031 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a3198dde-3bee-4659-b217-35286c1c7611-metrics-server-audit-profiles\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.208221 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.208076 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3198dde-3bee-4659-b217-35286c1c7611-client-ca-bundle\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.208221 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.208135 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a3198dde-3bee-4659-b217-35286c1c7611-secret-metrics-server-tls\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.208330 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.208306 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3198dde-3bee-4659-b217-35286c1c7611-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.309676 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.309591 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a3198dde-3bee-4659-b217-35286c1c7611-secret-metrics-server-tls\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.309841 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.309690 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3198dde-3bee-4659-b217-35286c1c7611-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.309841 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.309730 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hdjr\" (UniqueName: \"kubernetes.io/projected/a3198dde-3bee-4659-b217-35286c1c7611-kube-api-access-7hdjr\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.309841 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.309757 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a3198dde-3bee-4659-b217-35286c1c7611-secret-metrics-server-client-certs\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.309841 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.309788 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a3198dde-3bee-4659-b217-35286c1c7611-audit-log\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.309841 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.309816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a3198dde-3bee-4659-b217-35286c1c7611-metrics-server-audit-profiles\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.310091 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.309852 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3198dde-3bee-4659-b217-35286c1c7611-client-ca-bundle\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.310379 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.310347 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a3198dde-3bee-4659-b217-35286c1c7611-audit-log\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.310719 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.310690 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3198dde-3bee-4659-b217-35286c1c7611-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.311155 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.311128 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a3198dde-3bee-4659-b217-35286c1c7611-metrics-server-audit-profiles\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.312715 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.312670 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a3198dde-3bee-4659-b217-35286c1c7611-secret-metrics-server-tls\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.312808 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.312737 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a3198dde-3bee-4659-b217-35286c1c7611-secret-metrics-server-client-certs\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.312923 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.312900 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3198dde-3bee-4659-b217-35286c1c7611-client-ca-bundle\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.319641 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.319622 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hdjr\" (UniqueName: \"kubernetes.io/projected/a3198dde-3bee-4659-b217-35286c1c7611-kube-api-access-7hdjr\") pod \"metrics-server-c89ffd77f-v4q2w\" (UID: \"a3198dde-3bee-4659-b217-35286c1c7611\") " pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.384784 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.384741 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:00.529380 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.529305 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:00.614429 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.614362 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:00.685691 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:00.685658 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:02.003392 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.003354 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:02.011662 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.011626 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:31:02.016305 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.016273 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.022077 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.021721 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 10 18:31:02.022077 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.021796 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 10 18:31:02.022077 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.021812 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-l5ms7\"" Jul 10 18:31:02.022077 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.021868 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 10 18:31:02.022077 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.021737 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-23sdhkjol8crs\"" Jul 10 18:31:02.022077 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.021950 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 10 18:31:02.022077 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.022046 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 10 18:31:02.022592 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.022246 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 10 18:31:02.022592 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.022346 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 10 18:31:02.022592 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.022423 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 10 18:31:02.023038 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.022884 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 10 18:31:02.023038 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.022932 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 10 18:31:02.023242 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.023171 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 10 18:31:02.023242 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.023236 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 10 18:31:02.023517 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.023496 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 10 18:31:02.024683 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.024662 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 10 18:31:02.033775 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.033739 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:31:02.075731 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.075687 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:02.126858 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.126820 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127045 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.126864 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127045 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.126919 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127045 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127030 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127172 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127065 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127172 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127127 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127296 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127170 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d99g4\" (UniqueName: \"kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-kube-api-access-d99g4\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127296 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127296 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127234 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127296 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127261 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127296 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127288 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127452 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127313 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-config\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127452 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127397 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127452 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127418 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127452 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127434 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127589 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127455 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127589 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127485 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127589 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127506 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127589 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127541 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.127589 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.127568 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228161 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228125 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228374 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228181 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d99g4\" (UniqueName: \"kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-kube-api-access-d99g4\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228374 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228229 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228374 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228374 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228279 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228625 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228600 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228682 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228660 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-config\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228746 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228729 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228796 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228757 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228796 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228780 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228893 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228801 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228893 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.228893 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228868 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.229030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228906 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.229030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228939 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.229030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.228991 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.229030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.229014 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.229258 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.229036 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.229258 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.229078 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.229258 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.229104 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.229415 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.229290 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.229415 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.229312 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.230224 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.229576 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.230224 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.229798 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.230224 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.229892 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.230722 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.230695 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.232766 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.232724 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.237412 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.233466 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.237412 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.233930 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.237412 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.233995 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.237412 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.235388 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.237412 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.235547 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.237412 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.237263 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.238078 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.237962 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.238078 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.238029 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.238078 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.238055 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.238357 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.238137 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-config\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.240275 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.240224 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d99g4\" (UniqueName: \"kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-kube-api-access-d99g4\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.240562 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.240520 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.241057 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.241038 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:02.331764 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:02.331672 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:03.138871 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:03.138841 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:31:03.303657 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:03.303610 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:03.406328 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:03.406232 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:03.803985 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:31:03.803948 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d9c257e_3e4b_4017_a462_6c37c213eddd.slice/crio-6571fd7f816f908aafb5ef3e351b79da0b6c9ddf251f9f397d4d0fe3170f00b9 WatchSource:0}: Error finding container 6571fd7f816f908aafb5ef3e351b79da0b6c9ddf251f9f397d4d0fe3170f00b9: Status 404 returned error can't find the container with id 6571fd7f816f908aafb5ef3e351b79da0b6c9ddf251f9f397d4d0fe3170f00b9 Jul 10 18:31:03.806444 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:03.806417 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:31:03.981166 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:03.981022 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c89ffd77f-v4q2w"] Jul 10 18:31:03.984092 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:31:03.984063 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3198dde_3bee_4659_b217_35286c1c7611.slice/crio-7b6fef05023c218333feb6cb169a9129fbebd21f1931bbc67abd8195ec35ef19 WatchSource:0}: Error finding container 7b6fef05023c218333feb6cb169a9129fbebd21f1931bbc67abd8195ec35ef19: Status 404 returned error can't find the container with id 7b6fef05023c218333feb6cb169a9129fbebd21f1931bbc67abd8195ec35ef19 Jul 10 18:31:03.987028 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:03.987002 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:31:03.994875 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:31:03.994840 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde6c1f0e_0c3c_49fe_adb4_a000a85ee516.slice/crio-a98b5ae04313c116d6f08c8960cc6e905019fda1fbd1e36698d71d4721e87a3b WatchSource:0}: Error finding container a98b5ae04313c116d6f08c8960cc6e905019fda1fbd1e36698d71d4721e87a3b: Status 404 returned error can't find the container with id a98b5ae04313c116d6f08c8960cc6e905019fda1fbd1e36698d71d4721e87a3b Jul 10 18:31:04.013469 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.013429 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:31:04.017597 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:31:04.017554 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a91daf7_9723_454e_bd23_baabdfbcd14a.slice/crio-b7d7a99d52eebe0b24e50c7b174627db3e2ed838d58e82826eefaf3b090804bd WatchSource:0}: Error finding container b7d7a99d52eebe0b24e50c7b174627db3e2ed838d58e82826eefaf3b090804bd: Status 404 returned error can't find the container with id b7d7a99d52eebe0b24e50c7b174627db3e2ed838d58e82826eefaf3b090804bd Jul 10 18:31:04.032720 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.032693 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6"] Jul 10 18:31:04.034585 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:31:04.034558 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9493119a_77ed_47bf_b4bc_2e1315e24e9b.slice/crio-4e43b09549216a0f8be78817a0c8383336da88fcb7c76a7cf310fd392546d5d9 WatchSource:0}: Error finding container 4e43b09549216a0f8be78817a0c8383336da88fcb7c76a7cf310fd392546d5d9: Status 404 returned error can't find the container with id 4e43b09549216a0f8be78817a0c8383336da88fcb7c76a7cf310fd392546d5d9 Jul 10 18:31:04.250348 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.250308 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerStarted","Data":"a98b5ae04313c116d6f08c8960cc6e905019fda1fbd1e36698d71d4721e87a3b"} Jul 10 18:31:04.251886 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.251858 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" event={"ID":"a3198dde-3bee-4659-b217-35286c1c7611","Type":"ContainerStarted","Data":"7b6fef05023c218333feb6cb169a9129fbebd21f1931bbc67abd8195ec35ef19"} Jul 10 18:31:04.254602 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.254578 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-hsvxt" event={"ID":"9c12efa8-f9e8-46a2-8ad3-3fa3e645dcde","Type":"ContainerStarted","Data":"147eb533b00b84464cefa0e10ee6714c391ae872fddeae44a1007fe40c3c40ac"} Jul 10 18:31:04.255117 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.255072 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-hsvxt" Jul 10 18:31:04.257124 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.257015 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ltvfr" event={"ID":"4d9c257e-3e4b-4017-a462-6c37c213eddd","Type":"ContainerStarted","Data":"6571fd7f816f908aafb5ef3e351b79da0b6c9ddf251f9f397d4d0fe3170f00b9"} Jul 10 18:31:04.258652 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.258581 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" event={"ID":"9493119a-77ed-47bf-b4bc-2e1315e24e9b","Type":"ContainerStarted","Data":"4e43b09549216a0f8be78817a0c8383336da88fcb7c76a7cf310fd392546d5d9"} Jul 10 18:31:04.260157 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.260087 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerStarted","Data":"b7d7a99d52eebe0b24e50c7b174627db3e2ed838d58e82826eefaf3b090804bd"} Jul 10 18:31:04.266581 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.266549 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-hsvxt" Jul 10 18:31:04.281382 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.281122 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-hsvxt" podStartSLOduration=1.06000418 podStartE2EDuration="20.2811s" totalImagesPullingTime="19.22109582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:31:04.274358327 +0000 UTC m=+309.480198381" watchObservedRunningTime="2026-07-10 18:31:04.2811 +0000 UTC m=+309.486940056" Jul 10 18:31:04.282501 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.282413 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:04.332662 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.332619 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-568dc68b94-sstf6"] Jul 10 18:31:04.349015 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.348957 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:04.992614 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.992544 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b97dbbcd-5qtqv"] Jul 10 18:31:04.998545 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:04.998385 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.014644 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.014573 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b97dbbcd-5qtqv"] Jul 10 18:31:05.015429 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.015047 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 18:31:05.163656 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.163333 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-config\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.163656 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.163421 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-serving-cert\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.163656 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.163458 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-trusted-ca-bundle\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.163656 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.163488 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-oauth-config\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.163656 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.163577 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-oauth-serving-cert\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.163656 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.163606 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n6md\" (UniqueName: \"kubernetes.io/projected/6f686b83-e37c-4b7e-9759-c6b56d56a99f-kube-api-access-6n6md\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.163656 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.163649 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-service-ca\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.264460 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.264378 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-service-ca\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.265530 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.264493 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-config\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.265530 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.264545 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-serving-cert\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.265530 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.264576 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-trusted-ca-bundle\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.265530 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.264605 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-oauth-config\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.265530 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.264660 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-oauth-serving-cert\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.265530 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.264687 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n6md\" (UniqueName: \"kubernetes.io/projected/6f686b83-e37c-4b7e-9759-c6b56d56a99f-kube-api-access-6n6md\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.265530 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.265172 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-service-ca\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.266356 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.265664 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-oauth-serving-cert\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.266356 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.265800 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-trusted-ca-bundle\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.266356 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.266174 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-config\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.268915 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.268881 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-serving-cert\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.270700 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.270659 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-oauth-config\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.273501 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.273365 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n6md\" (UniqueName: \"kubernetes.io/projected/6f686b83-e37c-4b7e-9759-c6b56d56a99f-kube-api-access-6n6md\") pod \"console-7b97dbbcd-5qtqv\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:05.291766 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.291736 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:31:05.318994 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:05.318961 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:06.602653 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:06.602614 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67b79b8bcc-fwqbw"] Jul 10 18:31:06.678469 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:06.678433 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:06.865872 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:06.865704 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:06.901310 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:06.901264 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b97dbbcd-5qtqv"] Jul 10 18:31:06.965979 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:06.965897 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:07.471680 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:31:07.471648 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f686b83_e37c_4b7e_9759_c6b56d56a99f.slice/crio-b0a918490d1fdfca28ee14c461759073ab6ae6f03ee75d5c3acf11bd3e97373e WatchSource:0}: Error finding container b0a918490d1fdfca28ee14c461759073ab6ae6f03ee75d5c3acf11bd3e97373e: Status 404 returned error can't find the container with id b0a918490d1fdfca28ee14c461759073ab6ae6f03ee75d5c3acf11bd3e97373e Jul 10 18:31:08.283291 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.283256 2565 generic.go:356] "Generic (PLEG): container finished" podID="4d9c257e-3e4b-4017-a462-6c37c213eddd" containerID="43c361f5780b4e5bd99248f767b7aa2fa428c4afc3f35babbf9f7cc5e761edc8" exitCode=0 Jul 10 18:31:08.283708 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.283365 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ltvfr" event={"ID":"4d9c257e-3e4b-4017-a462-6c37c213eddd","Type":"ContainerDied","Data":"43c361f5780b4e5bd99248f767b7aa2fa428c4afc3f35babbf9f7cc5e761edc8"} Jul 10 18:31:08.286791 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.286426 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" event={"ID":"9493119a-77ed-47bf-b4bc-2e1315e24e9b","Type":"ContainerStarted","Data":"b1f5bb396021ff8aa63d040a6f27711ce1f1c6d9d5b9d8e5e2acc09b04110aa4"} Jul 10 18:31:08.286791 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.286499 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" event={"ID":"9493119a-77ed-47bf-b4bc-2e1315e24e9b","Type":"ContainerStarted","Data":"9d29c16b0fb5abae99b2d397a2ea995de780091ed7613e8f089784cb6cbb3fa5"} Jul 10 18:31:08.289068 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.289042 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b97dbbcd-5qtqv" event={"ID":"6f686b83-e37c-4b7e-9759-c6b56d56a99f","Type":"ContainerStarted","Data":"bfedb79e8245ce7e02da03116dfcaea9da3232cadd269db9d91a8879c6e2334f"} Jul 10 18:31:08.289217 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.289093 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b97dbbcd-5qtqv" event={"ID":"6f686b83-e37c-4b7e-9759-c6b56d56a99f","Type":"ContainerStarted","Data":"b0a918490d1fdfca28ee14c461759073ab6ae6f03ee75d5c3acf11bd3e97373e"} Jul 10 18:31:08.290888 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.290862 2565 generic.go:356] "Generic (PLEG): container finished" podID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerID="fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550" exitCode=0 Jul 10 18:31:08.290993 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.290943 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerDied","Data":"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550"} Jul 10 18:31:08.293152 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.293125 2565 generic.go:356] "Generic (PLEG): container finished" podID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerID="839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572" exitCode=0 Jul 10 18:31:08.293471 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.293451 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerDied","Data":"839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572"} Jul 10 18:31:08.295667 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.295641 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" event={"ID":"a3198dde-3bee-4659-b217-35286c1c7611","Type":"ContainerStarted","Data":"389ca29fa162fa0c1f294a6d72a11c98cc0f069d64355a36b541f26f5d122f55"} Jul 10 18:31:08.343012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.342962 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" podStartSLOduration=5.588048375 podStartE2EDuration="8.342947271s" totalImagesPullingTime="2.754898896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:31:08.342062047 +0000 UTC m=+313.547902127" watchObservedRunningTime="2026-07-10 18:31:08.342947271 +0000 UTC m=+313.548787326" Jul 10 18:31:08.387564 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:08.387507 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b97dbbcd-5qtqv" podStartSLOduration=4.387486453 podStartE2EDuration="4.387486453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:31:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:31:08.38674994 +0000 UTC m=+313.592589995" watchObservedRunningTime="2026-07-10 18:31:08.387486453 +0000 UTC m=+313.593326508" Jul 10 18:31:09.321583 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:09.321481 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ltvfr" event={"ID":"4d9c257e-3e4b-4017-a462-6c37c213eddd","Type":"ContainerStarted","Data":"eef00e9fe49bfd0cd8b189ae3a0c86ea23e4d04aeec9d40a83f05e45fd2933d8"} Jul 10 18:31:09.321583 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:09.321546 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ltvfr" event={"ID":"4d9c257e-3e4b-4017-a462-6c37c213eddd","Type":"ContainerStarted","Data":"b29d5c07e1776619f8d244a2d0af3388aab16003e982a5398d24bbf584e96161"} Jul 10 18:31:09.327721 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:09.327691 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" event={"ID":"9493119a-77ed-47bf-b4bc-2e1315e24e9b","Type":"ContainerStarted","Data":"8d625adf182b3e2ccacd7b2d202fd4897e7fea0661ac02894fa484b2ceb44f3a"} Jul 10 18:31:09.348599 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:09.348541 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ltvfr" podStartSLOduration=11.294371497 podStartE2EDuration="14.34851944s" totalImagesPullingTime="2.934030616s" totalInitContainerRuntime="120.117327ms" isStatefulPod=true podCreationTimestamp="2026-07-10 18:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:31:09.345743309 +0000 UTC m=+314.551583371" watchObservedRunningTime="2026-07-10 18:31:09.34851944 +0000 UTC m=+314.554359497" Jul 10 18:31:10.013972 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.013922 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b97dbbcd-5qtqv"] Jul 10 18:31:10.043213 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.043158 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74b7d5dddf-dzqgn"] Jul 10 18:31:10.067470 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.067408 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:10.080971 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.080934 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74b7d5dddf-dzqgn"] Jul 10 18:31:10.081239 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.081056 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.127868 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.127829 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:10.221743 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.221666 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-serving-cert\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.221743 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.221715 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzsn9\" (UniqueName: \"kubernetes.io/projected/996c457d-48e9-4045-bded-270d270dd4e0-kube-api-access-rzsn9\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.222038 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.221863 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-console-config\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.222090 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.222057 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-trusted-ca-bundle\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.222142 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.222117 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-oauth-config\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.222211 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.222143 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-service-ca\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.222275 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.222211 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-oauth-serving-cert\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.323651 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.323615 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-serving-cert\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.324119 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.323666 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzsn9\" (UniqueName: \"kubernetes.io/projected/996c457d-48e9-4045-bded-270d270dd4e0-kube-api-access-rzsn9\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.324119 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.323696 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-console-config\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.324119 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.323818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-trusted-ca-bundle\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.324119 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.323872 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-oauth-config\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.324119 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.323895 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-service-ca\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.324119 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.323941 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-oauth-serving-cert\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.326228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.324830 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-service-ca\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.326228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.325880 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-console-config\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.326228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.326022 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-oauth-serving-cert\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.326228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.326221 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-trusted-ca-bundle\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.328506 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.328480 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-oauth-config\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.329853 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.329818 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-serving-cert\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.336274 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.336229 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzsn9\" (UniqueName: \"kubernetes.io/projected/996c457d-48e9-4045-bded-270d270dd4e0-kube-api-access-rzsn9\") pod \"console-74b7d5dddf-dzqgn\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.395047 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.394825 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:10.579296 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:10.579266 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74b7d5dddf-dzqgn"] Jul 10 18:31:10.700108 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:31:10.700065 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod996c457d_48e9_4045_bded_270d270dd4e0.slice/crio-fd4d0ac8fa0123613f63f1e1828207f937ab96e6688958123ab6db9aeeae6ee3 WatchSource:0}: Error finding container fd4d0ac8fa0123613f63f1e1828207f937ab96e6688958123ab6db9aeeae6ee3: Status 404 returned error can't find the container with id fd4d0ac8fa0123613f63f1e1828207f937ab96e6688958123ab6db9aeeae6ee3 Jul 10 18:31:11.344584 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:11.344542 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b7d5dddf-dzqgn" event={"ID":"996c457d-48e9-4045-bded-270d270dd4e0","Type":"ContainerStarted","Data":"bdcdb0c93bc32ccda377cd9ea99cdda343af9d454e52a24ed0c2a84f7506cf8d"} Jul 10 18:31:11.345107 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:11.344606 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b7d5dddf-dzqgn" event={"ID":"996c457d-48e9-4045-bded-270d270dd4e0","Type":"ContainerStarted","Data":"fd4d0ac8fa0123613f63f1e1828207f937ab96e6688958123ab6db9aeeae6ee3"} Jul 10 18:31:11.349648 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:11.349491 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" event={"ID":"9493119a-77ed-47bf-b4bc-2e1315e24e9b","Type":"ContainerStarted","Data":"adf800128d3944f3f2cfe008feb4383eea4287b0dd8e9f9ad21c2821725de7f6"} Jul 10 18:31:11.349648 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:11.349574 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" event={"ID":"9493119a-77ed-47bf-b4bc-2e1315e24e9b","Type":"ContainerStarted","Data":"440fb7432c9c7ce768915c45898af9da83e94991d72704dc1a004436796c02dd"} Jul 10 18:31:11.352078 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:11.351990 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerStarted","Data":"7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22"} Jul 10 18:31:11.365792 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:11.364902 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74b7d5dddf-dzqgn" podStartSLOduration=1.364874565 podStartE2EDuration="1.364874565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:31:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:31:11.363165747 +0000 UTC m=+316.569005825" watchObservedRunningTime="2026-07-10 18:31:11.364874565 +0000 UTC m=+316.570714621" Jul 10 18:31:13.369610 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.369578 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" event={"ID":"9493119a-77ed-47bf-b4bc-2e1315e24e9b","Type":"ContainerStarted","Data":"5f3b37d669d8ed05eb8fa56351975ec5cc8cc41db56d524d5e5d81520989136d"} Jul 10 18:31:13.370011 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.369748 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:31:13.373321 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.373288 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerStarted","Data":"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b"} Jul 10 18:31:13.373434 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.373365 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerStarted","Data":"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966"} Jul 10 18:31:13.373434 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.373408 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerStarted","Data":"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625"} Jul 10 18:31:13.373535 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.373447 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerStarted","Data":"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b"} Jul 10 18:31:13.373535 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.373485 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerStarted","Data":"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc"} Jul 10 18:31:13.376629 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.376609 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" Jul 10 18:31:13.377258 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.377232 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerStarted","Data":"04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c"} Jul 10 18:31:13.377349 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.377300 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerStarted","Data":"b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4"} Jul 10 18:31:13.377349 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.377341 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerStarted","Data":"bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187"} Jul 10 18:31:13.377416 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.377384 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerStarted","Data":"ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1"} Jul 10 18:31:13.377453 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.377414 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerStarted","Data":"d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c"} Jul 10 18:31:13.398292 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.398240 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7bf7c7f8f7-rlpl6" podStartSLOduration=9.549005211 podStartE2EDuration="15.398220348s" totalImagesPullingTime="5.849215137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:31:13.398136 +0000 UTC m=+318.603976056" watchObservedRunningTime="2026-07-10 18:31:13.398220348 +0000 UTC m=+318.604060424" Jul 10 18:31:13.437625 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.437568 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=11.361707112 podStartE2EDuration="17.43754579s" totalImagesPullingTime="6.053679667s" totalInitContainerRuntime="22.159011ms" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:31:13.435967755 +0000 UTC m=+318.641807810" watchObservedRunningTime="2026-07-10 18:31:13.43754579 +0000 UTC m=+318.643385849" Jul 10 18:31:13.919566 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:13.919520 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:14.020503 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:14.020466 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:14.385524 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:14.385486 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerStarted","Data":"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e"} Jul 10 18:31:14.430429 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:14.430377 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.297063205 podStartE2EDuration="13.430357584s" totalImagesPullingTime="7.121573128s" totalInitContainerRuntime="11.721251ms" isStatefulPod=false podCreationTimestamp="2026-07-10 18:31:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:31:14.429216884 +0000 UTC m=+319.635056959" watchObservedRunningTime="2026-07-10 18:31:14.430357584 +0000 UTC m=+319.636197617" Jul 10 18:31:15.319530 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:15.319494 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:16.243533 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:16.243482 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:16.338603 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:16.338566 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:17.332084 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:17.332030 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:31:19.196970 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:19.196930 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:19.344526 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:19.344489 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:20.385716 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:20.385679 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:20.386147 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:20.385743 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:20.395267 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:20.395243 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:20.395408 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:20.395301 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:20.399882 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:20.399862 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:20.417097 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:20.417071 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:31:20.698334 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:20.698247 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:20.802981 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:20.802933 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:20.899568 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:20.899536 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:23.000471 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:23.000428 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:23.073029 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:23.072980 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:27.449794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:27.449765 2565 generic.go:356] "Generic (PLEG): container finished" podID="aacd5c76-7b3e-4ad4-817c-279d66dae83e" containerID="43785e1fd6e234a3839587493967b90ec3c94dd2b3909210bafa2e4aa25bb66d" exitCode=0 Jul 10 18:31:27.450155 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:27.449840 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" event={"ID":"aacd5c76-7b3e-4ad4-817c-279d66dae83e","Type":"ContainerDied","Data":"43785e1fd6e234a3839587493967b90ec3c94dd2b3909210bafa2e4aa25bb66d"} Jul 10 18:31:27.450219 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:27.450202 2565 scope.go:117] "RemoveContainer" containerID="43785e1fd6e234a3839587493967b90ec3c94dd2b3909210bafa2e4aa25bb66d" Jul 10 18:31:27.960495 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:27.960468 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5ft29_06300a4b-433b-4fa0-b969-78d216179a3b/dns/0.log" Jul 10 18:31:27.967980 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:27.967954 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5ft29_06300a4b-433b-4fa0-b969-78d216179a3b/kube-rbac-proxy/0.log" Jul 10 18:31:28.057155 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:28.057129 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7bh5c_f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694/dns-node-resolver/0.log" Jul 10 18:31:28.457422 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:28.457391 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-57kb4" event={"ID":"aacd5c76-7b3e-4ad4-817c-279d66dae83e","Type":"ContainerStarted","Data":"e8de9f56648d4293d399d044a28bea5415f190a725a8fa64609b9468243e792c"} Jul 10 18:31:28.472794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:28.472744 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:28.574140 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:28.574099 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:29.364468 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.364420 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-568dc68b94-sstf6" podUID="fcd36080-d958-459c-96d2-8157766352cd" containerName="console" containerID="cri-o://c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658" gracePeriod=15 Jul 10 18:31:29.643483 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.643404 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:31:29.720782 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.720746 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-oauth-serving-cert\") pod \"fcd36080-d958-459c-96d2-8157766352cd\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " Jul 10 18:31:29.720782 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.720787 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-oauth-config\") pod \"fcd36080-d958-459c-96d2-8157766352cd\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " Jul 10 18:31:29.721032 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.720822 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-service-ca\") pod \"fcd36080-d958-459c-96d2-8157766352cd\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " Jul 10 18:31:29.721032 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.720852 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bxwl\" (UniqueName: \"kubernetes.io/projected/fcd36080-d958-459c-96d2-8157766352cd-kube-api-access-5bxwl\") pod \"fcd36080-d958-459c-96d2-8157766352cd\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " Jul 10 18:31:29.721032 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.720881 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-serving-cert\") pod \"fcd36080-d958-459c-96d2-8157766352cd\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " Jul 10 18:31:29.721032 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.720902 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-console-config\") pod \"fcd36080-d958-459c-96d2-8157766352cd\" (UID: \"fcd36080-d958-459c-96d2-8157766352cd\") " Jul 10 18:31:29.721329 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.721306 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-service-ca" (OuterVolumeSpecName: "service-ca") pod "fcd36080-d958-459c-96d2-8157766352cd" (UID: "fcd36080-d958-459c-96d2-8157766352cd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:31:29.721405 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.721383 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fcd36080-d958-459c-96d2-8157766352cd" (UID: "fcd36080-d958-459c-96d2-8157766352cd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:31:29.721405 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.721392 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-console-config" (OuterVolumeSpecName: "console-config") pod "fcd36080-d958-459c-96d2-8157766352cd" (UID: "fcd36080-d958-459c-96d2-8157766352cd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:31:29.723163 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.723134 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcd36080-d958-459c-96d2-8157766352cd-kube-api-access-5bxwl" (OuterVolumeSpecName: "kube-api-access-5bxwl") pod "fcd36080-d958-459c-96d2-8157766352cd" (UID: "fcd36080-d958-459c-96d2-8157766352cd"). InnerVolumeSpecName "kube-api-access-5bxwl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:31:29.723356 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.723206 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fcd36080-d958-459c-96d2-8157766352cd" (UID: "fcd36080-d958-459c-96d2-8157766352cd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:31:29.723356 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.723238 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fcd36080-d958-459c-96d2-8157766352cd" (UID: "fcd36080-d958-459c-96d2-8157766352cd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:31:29.822537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.822501 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-oauth-serving-cert\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:29.822537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.822533 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-oauth-config\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:29.822537 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.822545 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-service-ca\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:29.822801 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.822559 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bxwl\" (UniqueName: \"kubernetes.io/projected/fcd36080-d958-459c-96d2-8157766352cd-kube-api-access-5bxwl\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:29.822801 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.822572 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcd36080-d958-459c-96d2-8157766352cd-console-serving-cert\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:29.822801 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:29.822583 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcd36080-d958-459c-96d2-8157766352cd-console-config\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:30.468097 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:30.468064 2565 generic.go:356] "Generic (PLEG): container finished" podID="fcd36080-d958-459c-96d2-8157766352cd" containerID="c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658" exitCode=0 Jul 10 18:31:30.468376 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:30.468107 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568dc68b94-sstf6" Jul 10 18:31:30.468376 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:30.468136 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568dc68b94-sstf6" event={"ID":"fcd36080-d958-459c-96d2-8157766352cd","Type":"ContainerDied","Data":"c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658"} Jul 10 18:31:30.468376 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:30.468178 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568dc68b94-sstf6" event={"ID":"fcd36080-d958-459c-96d2-8157766352cd","Type":"ContainerDied","Data":"74da8aeefcf76e87405b1ccc37219c34ac9cbb2ccd8397f88700e029d922374b"} Jul 10 18:31:30.468376 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:30.468216 2565 scope.go:117] "RemoveContainer" containerID="c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658" Jul 10 18:31:30.477319 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:30.477296 2565 scope.go:117] "RemoveContainer" containerID="c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658" Jul 10 18:31:30.477617 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:31:30.477593 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658\": container with ID starting with c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658 not found: ID does not exist" containerID="c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658" Jul 10 18:31:30.477695 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:30.477630 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658"} err="failed to get container status \"c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658\": rpc error: code = NotFound desc = could not find container \"c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658\": container with ID starting with c22d2be29082b15e97a999e8e6e3e1b2f78eec47273c6bf9408e666247266658 not found: ID does not exist" Jul 10 18:31:30.499134 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:30.499090 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-568dc68b94-sstf6"] Jul 10 18:31:30.500636 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:30.500614 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-568dc68b94-sstf6"] Jul 10 18:31:31.294349 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:31.294317 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcd36080-d958-459c-96d2-8157766352cd" path="/var/lib/kubelet/pods/fcd36080-d958-459c-96d2-8157766352cd/volumes" Jul 10 18:31:31.630452 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:31.630420 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" podUID="2f08196c-7a64-43e1-9dd4-bf32a34b2844" containerName="registry" containerID="cri-o://39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f" gracePeriod=30 Jul 10 18:31:31.925494 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:31.925461 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:31:32.046512 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.046476 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnwp4\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-kube-api-access-mnwp4\") pod \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " Jul 10 18:31:32.046763 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.046534 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f08196c-7a64-43e1-9dd4-bf32a34b2844-ca-trust-extracted\") pod \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " Jul 10 18:31:32.046763 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.046583 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-image-registry-private-configuration\") pod \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " Jul 10 18:31:32.046763 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.046610 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-certificates\") pod \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " Jul 10 18:31:32.046763 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.046640 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-installation-pull-secrets\") pod \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " Jul 10 18:31:32.046763 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.046655 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-bound-sa-token\") pod \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " Jul 10 18:31:32.046763 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.046709 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-trusted-ca\") pod \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " Jul 10 18:31:32.046763 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.046752 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") pod \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\" (UID: \"2f08196c-7a64-43e1-9dd4-bf32a34b2844\") " Jul 10 18:31:32.047161 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.047054 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2f08196c-7a64-43e1-9dd4-bf32a34b2844" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:31:32.047245 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.047183 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2f08196c-7a64-43e1-9dd4-bf32a34b2844" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:31:32.049241 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.049213 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2f08196c-7a64-43e1-9dd4-bf32a34b2844" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:31:32.049384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.049216 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-kube-api-access-mnwp4" (OuterVolumeSpecName: "kube-api-access-mnwp4") pod "2f08196c-7a64-43e1-9dd4-bf32a34b2844" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844"). InnerVolumeSpecName "kube-api-access-mnwp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:31:32.049384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.049367 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2f08196c-7a64-43e1-9dd4-bf32a34b2844" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:31:32.049497 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.049459 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2f08196c-7a64-43e1-9dd4-bf32a34b2844" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:31:32.049727 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.049709 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2f08196c-7a64-43e1-9dd4-bf32a34b2844" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:31:32.055846 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.055817 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f08196c-7a64-43e1-9dd4-bf32a34b2844-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2f08196c-7a64-43e1-9dd4-bf32a34b2844" (UID: "2f08196c-7a64-43e1-9dd4-bf32a34b2844"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:31:32.147942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.147858 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-trusted-ca\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:32.147942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.147887 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-tls\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:32.147942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.147897 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnwp4\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-kube-api-access-mnwp4\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:32.147942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.147906 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f08196c-7a64-43e1-9dd4-bf32a34b2844-ca-trust-extracted\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:32.147942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.147915 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-image-registry-private-configuration\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:32.147942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.147924 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f08196c-7a64-43e1-9dd4-bf32a34b2844-registry-certificates\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:32.147942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.147934 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f08196c-7a64-43e1-9dd4-bf32a34b2844-installation-pull-secrets\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:32.147942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.147941 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f08196c-7a64-43e1-9dd4-bf32a34b2844-bound-sa-token\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:32.480542 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.480445 2565 generic.go:356] "Generic (PLEG): container finished" podID="2f08196c-7a64-43e1-9dd4-bf32a34b2844" containerID="39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f" exitCode=0 Jul 10 18:31:32.480542 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.480526 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" event={"ID":"2f08196c-7a64-43e1-9dd4-bf32a34b2844","Type":"ContainerDied","Data":"39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f"} Jul 10 18:31:32.481049 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.480554 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" Jul 10 18:31:32.481049 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.480582 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67b79b8bcc-fwqbw" event={"ID":"2f08196c-7a64-43e1-9dd4-bf32a34b2844","Type":"ContainerDied","Data":"ff1174d051dee9a6179b71a835377782afaed28516efcaca32e26af6a7f314a6"} Jul 10 18:31:32.481049 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.480614 2565 scope.go:117] "RemoveContainer" containerID="39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f" Jul 10 18:31:32.489920 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.489902 2565 scope.go:117] "RemoveContainer" containerID="39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f" Jul 10 18:31:32.490180 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:31:32.490162 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f\": container with ID starting with 39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f not found: ID does not exist" containerID="39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f" Jul 10 18:31:32.490287 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.490211 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f"} err="failed to get container status \"39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f\": rpc error: code = NotFound desc = could not find container \"39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f\": container with ID starting with 39cadbbff65c362dd607e40c176f401d33b35e93064440c4bcffd6a1d5e4431f not found: ID does not exist" Jul 10 18:31:32.504856 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.504826 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67b79b8bcc-fwqbw"] Jul 10 18:31:32.509513 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:32.509488 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67b79b8bcc-fwqbw"] Jul 10 18:31:33.294465 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:33.294432 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f08196c-7a64-43e1-9dd4-bf32a34b2844" path="/var/lib/kubelet/pods/2f08196c-7a64-43e1-9dd4-bf32a34b2844/volumes" Jul 10 18:31:34.075005 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:34.074959 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:34.172412 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:34.172375 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:35.376037 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.375993 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b97dbbcd-5qtqv" podUID="6f686b83-e37c-4b7e-9759-c6b56d56a99f" containerName="console" containerID="cri-o://bfedb79e8245ce7e02da03116dfcaea9da3232cadd269db9d91a8879c6e2334f" gracePeriod=15 Jul 10 18:31:35.501937 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.501905 2565 generic.go:356] "Generic (PLEG): container finished" podID="6f686b83-e37c-4b7e-9759-c6b56d56a99f" containerID="bfedb79e8245ce7e02da03116dfcaea9da3232cadd269db9d91a8879c6e2334f" exitCode=0 Jul 10 18:31:35.502077 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.501982 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b97dbbcd-5qtqv" event={"ID":"6f686b83-e37c-4b7e-9759-c6b56d56a99f","Type":"ContainerDied","Data":"bfedb79e8245ce7e02da03116dfcaea9da3232cadd269db9d91a8879c6e2334f"} Jul 10 18:31:35.622875 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.622849 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:35.681886 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.681788 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-trusted-ca-bundle\") pod \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " Jul 10 18:31:35.681886 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.681838 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-config\") pod \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " Jul 10 18:31:35.681886 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.681864 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-oauth-config\") pod \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " Jul 10 18:31:35.682106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.681903 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n6md\" (UniqueName: \"kubernetes.io/projected/6f686b83-e37c-4b7e-9759-c6b56d56a99f-kube-api-access-6n6md\") pod \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " Jul 10 18:31:35.682106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.681960 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-serving-cert\") pod \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " Jul 10 18:31:35.682106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.682035 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-oauth-serving-cert\") pod \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " Jul 10 18:31:35.682106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.682057 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-service-ca\") pod \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\" (UID: \"6f686b83-e37c-4b7e-9759-c6b56d56a99f\") " Jul 10 18:31:35.682433 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.682297 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6f686b83-e37c-4b7e-9759-c6b56d56a99f" (UID: "6f686b83-e37c-4b7e-9759-c6b56d56a99f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:31:35.682547 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.682497 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6f686b83-e37c-4b7e-9759-c6b56d56a99f" (UID: "6f686b83-e37c-4b7e-9759-c6b56d56a99f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:31:35.682638 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.682522 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-config" (OuterVolumeSpecName: "console-config") pod "6f686b83-e37c-4b7e-9759-c6b56d56a99f" (UID: "6f686b83-e37c-4b7e-9759-c6b56d56a99f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:31:35.682702 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.682686 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-service-ca" (OuterVolumeSpecName: "service-ca") pod "6f686b83-e37c-4b7e-9759-c6b56d56a99f" (UID: "6f686b83-e37c-4b7e-9759-c6b56d56a99f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:31:35.684042 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.684015 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6f686b83-e37c-4b7e-9759-c6b56d56a99f" (UID: "6f686b83-e37c-4b7e-9759-c6b56d56a99f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:31:35.684223 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.684179 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f686b83-e37c-4b7e-9759-c6b56d56a99f-kube-api-access-6n6md" (OuterVolumeSpecName: "kube-api-access-6n6md") pod "6f686b83-e37c-4b7e-9759-c6b56d56a99f" (UID: "6f686b83-e37c-4b7e-9759-c6b56d56a99f"). InnerVolumeSpecName "kube-api-access-6n6md". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:31:35.684223 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.684187 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6f686b83-e37c-4b7e-9759-c6b56d56a99f" (UID: "6f686b83-e37c-4b7e-9759-c6b56d56a99f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:31:35.782968 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.782930 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-serving-cert\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:35.782968 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.782963 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-oauth-serving-cert\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:35.782968 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.782974 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-service-ca\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:35.783224 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.782982 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-trusted-ca-bundle\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:35.783224 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.782991 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-config\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:35.783224 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.783000 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f686b83-e37c-4b7e-9759-c6b56d56a99f-console-oauth-config\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:35.783224 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:35.783012 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n6md\" (UniqueName: \"kubernetes.io/projected/6f686b83-e37c-4b7e-9759-c6b56d56a99f-kube-api-access-6n6md\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:31:36.507800 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:36.507766 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b97dbbcd-5qtqv" Jul 10 18:31:36.507800 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:36.507779 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b97dbbcd-5qtqv" event={"ID":"6f686b83-e37c-4b7e-9759-c6b56d56a99f","Type":"ContainerDied","Data":"b0a918490d1fdfca28ee14c461759073ab6ae6f03ee75d5c3acf11bd3e97373e"} Jul 10 18:31:36.508359 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:36.507836 2565 scope.go:117] "RemoveContainer" containerID="bfedb79e8245ce7e02da03116dfcaea9da3232cadd269db9d91a8879c6e2334f" Jul 10 18:31:36.532952 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:36.532917 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b97dbbcd-5qtqv"] Jul 10 18:31:36.543379 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:36.543352 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b97dbbcd-5qtqv"] Jul 10 18:31:37.294157 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:37.294125 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f686b83-e37c-4b7e-9759-c6b56d56a99f" path="/var/lib/kubelet/pods/6f686b83-e37c-4b7e-9759-c6b56d56a99f/volumes" Jul 10 18:31:40.391043 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:40.391014 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:40.395209 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:40.395162 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-c89ffd77f-v4q2w" Jul 10 18:31:42.544107 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:42.544075 2565 generic.go:356] "Generic (PLEG): container finished" podID="9b56e500-4923-4ab3-8b46-052e2a42839b" containerID="1a226c78fbfcde27f98c88b7b730d36661422baa19ca30530e84eb9e26c7504f" exitCode=0 Jul 10 18:31:42.544511 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:42.544145 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" event={"ID":"9b56e500-4923-4ab3-8b46-052e2a42839b","Type":"ContainerDied","Data":"1a226c78fbfcde27f98c88b7b730d36661422baa19ca30530e84eb9e26c7504f"} Jul 10 18:31:42.544511 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:42.544502 2565 scope.go:117] "RemoveContainer" containerID="1a226c78fbfcde27f98c88b7b730d36661422baa19ca30530e84eb9e26c7504f" Jul 10 18:31:43.550479 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:43.550443 2565 generic.go:356] "Generic (PLEG): container finished" podID="d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d" containerID="64583fce3f56e698d3925f2b60465f6a6e58e7676f85ae613770e1ab38cd6b5f" exitCode=0 Jul 10 18:31:43.550953 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:43.550516 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" event={"ID":"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d","Type":"ContainerDied","Data":"64583fce3f56e698d3925f2b60465f6a6e58e7676f85ae613770e1ab38cd6b5f"} Jul 10 18:31:43.550953 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:43.550838 2565 scope.go:117] "RemoveContainer" containerID="64583fce3f56e698d3925f2b60465f6a6e58e7676f85ae613770e1ab38cd6b5f" Jul 10 18:31:43.552409 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:43.552389 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6kgms" event={"ID":"9b56e500-4923-4ab3-8b46-052e2a42839b","Type":"ContainerStarted","Data":"62f3e491f99b73f3e09327b715784c5b42db193e90ee3ffa4e5ff880c0e39361"} Jul 10 18:31:44.559720 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:44.559685 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-25szj" event={"ID":"d43482b7-b9be-4ecf-b5fb-fe276e9c8a3d","Type":"ContainerStarted","Data":"d6a3ffd00e24b5f3896d9a1611942a99f9265161c8d040c2d090d6905ab9d67e"} Jul 10 18:31:49.065142 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:49.065036 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:49.137119 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:31:49.137084 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:02.332943 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:02.332908 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:02.352765 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:02.352735 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:02.658539 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:02.658510 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:09.213637 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:09.213590 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:32:09.215949 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:09.215914 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908795-9d74-4fec-bb0b-859f3b202d44-metrics-certs\") pod \"network-metrics-daemon-tvbm5\" (UID: \"24908795-9d74-4fec-bb0b-859f3b202d44\") " pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:32:09.496679 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:09.496595 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fnqlf\"" Jul 10 18:32:09.503362 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:09.503333 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvbm5" Jul 10 18:32:09.648433 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:09.648404 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tvbm5"] Jul 10 18:32:09.650283 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:32:09.650258 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24908795_9d74_4fec_bb0b_859f3b202d44.slice/crio-dbaa090631f6023eb67f9f799f0a994ef7396e46cde971ae95b23cf06d33da6d WatchSource:0}: Error finding container dbaa090631f6023eb67f9f799f0a994ef7396e46cde971ae95b23cf06d33da6d: Status 404 returned error can't find the container with id dbaa090631f6023eb67f9f799f0a994ef7396e46cde971ae95b23cf06d33da6d Jul 10 18:32:09.669721 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:09.669687 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvbm5" event={"ID":"24908795-9d74-4fec-bb0b-859f3b202d44","Type":"ContainerStarted","Data":"dbaa090631f6023eb67f9f799f0a994ef7396e46cde971ae95b23cf06d33da6d"} Jul 10 18:32:11.680669 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:11.680628 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvbm5" event={"ID":"24908795-9d74-4fec-bb0b-859f3b202d44","Type":"ContainerStarted","Data":"148f32ab7d7d8b5a08cda3a0143ce841af4f2f9934c1b37d32b630be1bbb82f6"} Jul 10 18:32:11.681038 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:11.680684 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvbm5" event={"ID":"24908795-9d74-4fec-bb0b-859f3b202d44","Type":"ContainerStarted","Data":"5c11b5fb04f9e30260ce60872a6263d5d16ee84e5838e62ded0f924bea0354c8"} Jul 10 18:32:11.713992 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:11.713939 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tvbm5" podStartSLOduration=374.388720416 podStartE2EDuration="6m15.713923057s" totalImagesPullingTime="1.325202641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:32:11.711656386 +0000 UTC m=+376.917496485" watchObservedRunningTime="2026-07-10 18:32:11.713923057 +0000 UTC m=+376.919763111" Jul 10 18:32:11.855558 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:11.855516 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:11.976611 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:11.976532 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:16.364513 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.364475 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:32:16.365028 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.364749 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="alertmanager" containerID="cri-o://7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22" gracePeriod=120 Jul 10 18:32:16.365028 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.364817 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy-metric" containerID="cri-o://b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4" gracePeriod=120 Jul 10 18:32:16.365028 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.364863 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy" containerID="cri-o://bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187" gracePeriod=120 Jul 10 18:32:16.365028 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.364883 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="prom-label-proxy" containerID="cri-o://04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c" gracePeriod=120 Jul 10 18:32:16.365028 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.364858 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="config-reloader" containerID="cri-o://d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c" gracePeriod=120 Jul 10 18:32:16.365360 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.364827 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy-web" containerID="cri-o://ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1" gracePeriod=120 Jul 10 18:32:16.705864 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.705780 2565 generic.go:356] "Generic (PLEG): container finished" podID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerID="04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c" exitCode=0 Jul 10 18:32:16.705864 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.705804 2565 generic.go:356] "Generic (PLEG): container finished" podID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerID="bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187" exitCode=0 Jul 10 18:32:16.705864 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.705811 2565 generic.go:356] "Generic (PLEG): container finished" podID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerID="d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c" exitCode=0 Jul 10 18:32:16.705864 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.705817 2565 generic.go:356] "Generic (PLEG): container finished" podID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerID="7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22" exitCode=0 Jul 10 18:32:16.705864 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.705850 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerDied","Data":"04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c"} Jul 10 18:32:16.706126 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.705914 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerDied","Data":"bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187"} Jul 10 18:32:16.706126 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.705941 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerDied","Data":"d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c"} Jul 10 18:32:16.706126 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:16.705965 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerDied","Data":"7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22"} Jul 10 18:32:17.617498 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.617475 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:32:17.689185 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689112 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-web-config\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.689185 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689150 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-volume\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.689185 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689172 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-web\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.690311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689235 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlkl8\" (UniqueName: \"kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-kube-api-access-jlkl8\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.690311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689296 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-tls-assets\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.690311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689323 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-cluster-tls-config\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.690311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689395 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-main-db\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.690311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689446 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-main-tls\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.690311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689475 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-metrics-client-ca\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.690311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689518 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-trusted-ca-bundle\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.690311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689567 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-out\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.690311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689593 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.690311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689616 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-metric\") pod \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\" (UID: \"de6c1f0e-0c3c-49fe-adb4-a000a85ee516\") " Jul 10 18:32:17.690311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.689903 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:32:17.695320 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.693365 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:32:17.695320 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.693573 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:32:17.697755 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.697710 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-out" (OuterVolumeSpecName: "config-out") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:32:17.699690 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.698805 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:17.699690 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.698942 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:17.699690 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.699012 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-volume" (OuterVolumeSpecName: "config-volume") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:17.699690 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.699046 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:32:17.699690 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.699076 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:17.699690 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.699665 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:17.702372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.700300 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-kube-api-access-jlkl8" (OuterVolumeSpecName: "kube-api-access-jlkl8") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "kube-api-access-jlkl8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:32:17.705429 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.705394 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:17.714168 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.714133 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-web-config" (OuterVolumeSpecName: "web-config") pod "de6c1f0e-0c3c-49fe-adb4-a000a85ee516" (UID: "de6c1f0e-0c3c-49fe-adb4-a000a85ee516"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:17.716676 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.716648 2565 generic.go:356] "Generic (PLEG): container finished" podID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerID="b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4" exitCode=0 Jul 10 18:32:17.716676 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.716674 2565 generic.go:356] "Generic (PLEG): container finished" podID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerID="ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1" exitCode=0 Jul 10 18:32:17.716848 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.716698 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerDied","Data":"b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4"} Jul 10 18:32:17.716848 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.716762 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:32:17.716848 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.716781 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerDied","Data":"ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1"} Jul 10 18:32:17.716848 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.716816 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de6c1f0e-0c3c-49fe-adb4-a000a85ee516","Type":"ContainerDied","Data":"a98b5ae04313c116d6f08c8960cc6e905019fda1fbd1e36698d71d4721e87a3b"} Jul 10 18:32:17.717055 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.716855 2565 scope.go:117] "RemoveContainer" containerID="04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c" Jul 10 18:32:17.726481 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.726408 2565 scope.go:117] "RemoveContainer" containerID="b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4" Jul 10 18:32:17.733952 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.733931 2565 scope.go:117] "RemoveContainer" containerID="bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187" Jul 10 18:32:17.741664 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.741647 2565 scope.go:117] "RemoveContainer" containerID="ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1" Jul 10 18:32:17.748494 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.748472 2565 scope.go:117] "RemoveContainer" containerID="d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c" Jul 10 18:32:17.753919 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.753881 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:32:17.757333 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.757233 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:32:17.757398 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.757382 2565 scope.go:117] "RemoveContainer" containerID="7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22" Jul 10 18:32:17.764549 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.764529 2565 scope.go:117] "RemoveContainer" containerID="839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572" Jul 10 18:32:17.771356 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.771339 2565 scope.go:117] "RemoveContainer" containerID="04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c" Jul 10 18:32:17.771607 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:17.771586 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c\": container with ID starting with 04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c not found: ID does not exist" containerID="04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c" Jul 10 18:32:17.771654 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.771615 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c"} err="failed to get container status \"04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c\": rpc error: code = NotFound desc = could not find container \"04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c\": container with ID starting with 04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c not found: ID does not exist" Jul 10 18:32:17.771654 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.771635 2565 scope.go:117] "RemoveContainer" containerID="b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4" Jul 10 18:32:17.771837 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:17.771818 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4\": container with ID starting with b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4 not found: ID does not exist" containerID="b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4" Jul 10 18:32:17.771936 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.771847 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4"} err="failed to get container status \"b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4\": rpc error: code = NotFound desc = could not find container \"b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4\": container with ID starting with b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4 not found: ID does not exist" Jul 10 18:32:17.771936 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.771871 2565 scope.go:117] "RemoveContainer" containerID="bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187" Jul 10 18:32:17.772137 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:17.772120 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187\": container with ID starting with bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187 not found: ID does not exist" containerID="bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187" Jul 10 18:32:17.772173 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.772141 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187"} err="failed to get container status \"bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187\": rpc error: code = NotFound desc = could not find container \"bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187\": container with ID starting with bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187 not found: ID does not exist" Jul 10 18:32:17.772173 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.772156 2565 scope.go:117] "RemoveContainer" containerID="ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1" Jul 10 18:32:17.772371 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:17.772354 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1\": container with ID starting with ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1 not found: ID does not exist" containerID="ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1" Jul 10 18:32:17.772410 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.772375 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1"} err="failed to get container status \"ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1\": rpc error: code = NotFound desc = could not find container \"ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1\": container with ID starting with ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1 not found: ID does not exist" Jul 10 18:32:17.772410 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.772387 2565 scope.go:117] "RemoveContainer" containerID="d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c" Jul 10 18:32:17.772578 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:17.772561 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c\": container with ID starting with d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c not found: ID does not exist" containerID="d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c" Jul 10 18:32:17.772622 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.772582 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c"} err="failed to get container status \"d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c\": rpc error: code = NotFound desc = could not find container \"d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c\": container with ID starting with d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c not found: ID does not exist" Jul 10 18:32:17.772622 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.772601 2565 scope.go:117] "RemoveContainer" containerID="7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22" Jul 10 18:32:17.772814 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:17.772799 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22\": container with ID starting with 7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22 not found: ID does not exist" containerID="7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22" Jul 10 18:32:17.772865 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.772819 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22"} err="failed to get container status \"7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22\": rpc error: code = NotFound desc = could not find container \"7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22\": container with ID starting with 7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22 not found: ID does not exist" Jul 10 18:32:17.772865 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.772835 2565 scope.go:117] "RemoveContainer" containerID="839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572" Jul 10 18:32:17.773080 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:17.773062 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572\": container with ID starting with 839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572 not found: ID does not exist" containerID="839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572" Jul 10 18:32:17.773120 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.773085 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572"} err="failed to get container status \"839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572\": rpc error: code = NotFound desc = could not find container \"839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572\": container with ID starting with 839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572 not found: ID does not exist" Jul 10 18:32:17.773120 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.773100 2565 scope.go:117] "RemoveContainer" containerID="04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c" Jul 10 18:32:17.773391 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.773372 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c"} err="failed to get container status \"04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c\": rpc error: code = NotFound desc = could not find container \"04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c\": container with ID starting with 04a653745dd1444e4828e30223d42ebc536cfa865f0176d17ac05e02a13ed95c not found: ID does not exist" Jul 10 18:32:17.773444 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.773391 2565 scope.go:117] "RemoveContainer" containerID="b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4" Jul 10 18:32:17.773616 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.773597 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4"} err="failed to get container status \"b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4\": rpc error: code = NotFound desc = could not find container \"b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4\": container with ID starting with b970c6114a036eb16f5a304aa4cfad8ddf6b1dbde03b9ddfeccb826257b56db4 not found: ID does not exist" Jul 10 18:32:17.773616 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.773614 2565 scope.go:117] "RemoveContainer" containerID="bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187" Jul 10 18:32:17.773832 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.773814 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187"} err="failed to get container status \"bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187\": rpc error: code = NotFound desc = could not find container \"bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187\": container with ID starting with bc229711441f362720e75274b89ef9384423e3c1b844231c1d5518f715366187 not found: ID does not exist" Jul 10 18:32:17.773832 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.773831 2565 scope.go:117] "RemoveContainer" containerID="ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1" Jul 10 18:32:17.774065 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.774047 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1"} err="failed to get container status \"ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1\": rpc error: code = NotFound desc = could not find container \"ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1\": container with ID starting with ece5105daa459c13d9009d5de503d79b905b7e260b13075bd472868797806ee1 not found: ID does not exist" Jul 10 18:32:17.774120 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.774066 2565 scope.go:117] "RemoveContainer" containerID="d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c" Jul 10 18:32:17.774311 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.774294 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c"} err="failed to get container status \"d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c\": rpc error: code = NotFound desc = could not find container \"d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c\": container with ID starting with d4b5a585539054cfc8fb341356388e510757732e6157bfdb8e3e42bb47f99b3c not found: ID does not exist" Jul 10 18:32:17.774385 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.774312 2565 scope.go:117] "RemoveContainer" containerID="7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22" Jul 10 18:32:17.774530 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.774513 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22"} err="failed to get container status \"7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22\": rpc error: code = NotFound desc = could not find container \"7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22\": container with ID starting with 7cdd6096d87adad6df273ec4ea6e0b093fb3a26b9fe8259a5e33027565251d22 not found: ID does not exist" Jul 10 18:32:17.774530 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.774530 2565 scope.go:117] "RemoveContainer" containerID="839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572" Jul 10 18:32:17.774728 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.774711 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572"} err="failed to get container status \"839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572\": rpc error: code = NotFound desc = could not find container \"839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572\": container with ID starting with 839959ffc016cabffce2198b24fe3f802f345d6c473569368e67322cf2d58572 not found: ID does not exist" Jul 10 18:32:17.790928 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.790891 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlkl8\" (UniqueName: \"kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-kube-api-access-jlkl8\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.790928 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.790921 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-tls-assets\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.791110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.790935 2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-cluster-tls-config\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.791110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.790945 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-main-db\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.791110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.790954 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-main-tls\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.791110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.790962 2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-metrics-client-ca\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.791110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.790970 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.791110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.790979 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-out\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.791110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.790991 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.791110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.791006 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.791110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.791045 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-web-config\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.791110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.791064 2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-config-volume\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:17.791110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:17.791076 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de6c1f0e-0c3c-49fe-adb4-a000a85ee516-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:19.007535 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:19.007500 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:19.294472 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:19.294395 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" path="/var/lib/kubelet/pods/de6c1f0e-0c3c-49fe-adb4-a000a85ee516/volumes" Jul 10 18:32:20.698989 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:20.698953 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:32:20.699523 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:20.699352 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="prometheus" containerID="cri-o://a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc" gracePeriod=600 Jul 10 18:32:20.699523 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:20.699469 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e" gracePeriod=600 Jul 10 18:32:20.699523 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:20.699508 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="config-reloader" containerID="cri-o://a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b" gracePeriod=600 Jul 10 18:32:20.699684 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:20.699531 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy-web" containerID="cri-o://df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966" gracePeriod=600 Jul 10 18:32:20.699684 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:20.699532 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy" containerID="cri-o://232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b" gracePeriod=600 Jul 10 18:32:20.699684 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:20.699505 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="thanos-sidecar" containerID="cri-o://67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625" gracePeriod=600 Jul 10 18:32:20.951528 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:20.951463 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.123053 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123021 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-metrics-client-certs\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123053 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123058 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-thanos-prometheus-http-client-file\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123095 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-tls-assets\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123115 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-kubelet-serving-ca-bundle\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123131 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-config\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123168 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-grpc-tls\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123208 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-web-config\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123247 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-config-out\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123292 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-db\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123337 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123329 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-2\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123375 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123403 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123438 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-0\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123473 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d99g4\" (UniqueName: \"kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-kube-api-access-d99g4\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123504 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-trusted-ca-bundle\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123520 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:32:21.123725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123536 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-tls\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123627 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-serving-certs-ca-bundle\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123668 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-kube-rbac-proxy\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123692 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-metrics-client-ca\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.123725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123725 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-1\") pod \"5a91daf7-9723-454e-bd23-baabdfbcd14a\" (UID: \"5a91daf7-9723-454e-bd23-baabdfbcd14a\") " Jul 10 18:32:21.124286 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.123778 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:32:21.124286 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.124021 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.124286 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.124042 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.124433 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.124317 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:32:21.124590 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.124565 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:32:21.125646 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.125618 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:21.125871 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.125844 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:21.125871 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.125863 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:32:21.126320 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.126294 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:32:21.127354 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.126840 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:21.127354 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.127118 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:21.127514 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.127376 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-config" (OuterVolumeSpecName: "config") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:21.127754 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.127722 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:32:21.127867 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.127767 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:32:21.127867 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.127768 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:32:21.127990 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.127894 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:21.128482 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.128456 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-config-out" (OuterVolumeSpecName: "config-out") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:32:21.128770 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.128749 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:21.129047 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.129029 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-kube-api-access-d99g4" (OuterVolumeSpecName: "kube-api-access-d99g4") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "kube-api-access-d99g4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:32:21.129553 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.129529 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:21.138047 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.138019 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-web-config" (OuterVolumeSpecName: "web-config") pod "5a91daf7-9723-454e-bd23-baabdfbcd14a" (UID: "5a91daf7-9723-454e-bd23-baabdfbcd14a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:32:21.224687 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224602 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-config-out\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.224687 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224631 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-db\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.224687 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224643 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.224687 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224655 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.224687 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224666 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.224687 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224674 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d99g4\" (UniqueName: \"kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-kube-api-access-d99g4\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.224687 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224684 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.224687 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224693 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.225013 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224702 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.225013 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224711 2565 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-kube-rbac-proxy\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.225013 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224719 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-configmap-metrics-client-ca\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.225013 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224727 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5a91daf7-9723-454e-bd23-baabdfbcd14a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.225013 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224737 2565 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-metrics-client-certs\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.225013 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224746 2565 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.225013 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224753 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5a91daf7-9723-454e-bd23-baabdfbcd14a-tls-assets\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.225013 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224762 2565 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-config\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.225013 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224770 2565 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-secret-grpc-tls\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.225013 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.224777 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5a91daf7-9723-454e-bd23-baabdfbcd14a-web-config\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:32:21.740216 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740169 2565 generic.go:356] "Generic (PLEG): container finished" podID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerID="41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e" exitCode=0 Jul 10 18:32:21.740216 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740208 2565 generic.go:356] "Generic (PLEG): container finished" podID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerID="232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b" exitCode=0 Jul 10 18:32:21.740216 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740215 2565 generic.go:356] "Generic (PLEG): container finished" podID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerID="df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966" exitCode=0 Jul 10 18:32:21.740216 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740222 2565 generic.go:356] "Generic (PLEG): container finished" podID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerID="67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625" exitCode=0 Jul 10 18:32:21.740216 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740227 2565 generic.go:356] "Generic (PLEG): container finished" podID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerID="a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b" exitCode=0 Jul 10 18:32:21.740741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740232 2565 generic.go:356] "Generic (PLEG): container finished" podID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerID="a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc" exitCode=0 Jul 10 18:32:21.740741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740254 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerDied","Data":"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e"} Jul 10 18:32:21.740741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740274 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.740741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740348 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerDied","Data":"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b"} Jul 10 18:32:21.740741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740376 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerDied","Data":"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966"} Jul 10 18:32:21.740741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740404 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerDied","Data":"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625"} Jul 10 18:32:21.740741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740422 2565 scope.go:117] "RemoveContainer" containerID="41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e" Jul 10 18:32:21.740741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740428 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerDied","Data":"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b"} Jul 10 18:32:21.740741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740525 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerDied","Data":"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc"} Jul 10 18:32:21.740741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.740565 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a91daf7-9723-454e-bd23-baabdfbcd14a","Type":"ContainerDied","Data":"b7d7a99d52eebe0b24e50c7b174627db3e2ed838d58e82826eefaf3b090804bd"} Jul 10 18:32:21.748154 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.748135 2565 scope.go:117] "RemoveContainer" containerID="232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b" Jul 10 18:32:21.756014 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.755997 2565 scope.go:117] "RemoveContainer" containerID="df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966" Jul 10 18:32:21.762386 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.762366 2565 scope.go:117] "RemoveContainer" containerID="67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625" Jul 10 18:32:21.769915 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.769889 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:32:21.770793 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.770772 2565 scope.go:117] "RemoveContainer" containerID="a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b" Jul 10 18:32:21.778226 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.778176 2565 scope.go:117] "RemoveContainer" containerID="a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc" Jul 10 18:32:21.779863 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.779847 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:32:21.785888 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.785873 2565 scope.go:117] "RemoveContainer" containerID="fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550" Jul 10 18:32:21.794401 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.794380 2565 scope.go:117] "RemoveContainer" containerID="41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e" Jul 10 18:32:21.794873 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:21.794851 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": container with ID starting with 41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e not found: ID does not exist" containerID="41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e" Jul 10 18:32:21.794956 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.794894 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e"} err="failed to get container status \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": rpc error: code = NotFound desc = could not find container \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": container with ID starting with 41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e not found: ID does not exist" Jul 10 18:32:21.794956 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.794919 2565 scope.go:117] "RemoveContainer" containerID="232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b" Jul 10 18:32:21.795256 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:21.795231 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": container with ID starting with 232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b not found: ID does not exist" containerID="232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b" Jul 10 18:32:21.795329 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.795256 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b"} err="failed to get container status \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": rpc error: code = NotFound desc = could not find container \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": container with ID starting with 232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b not found: ID does not exist" Jul 10 18:32:21.795329 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.795271 2565 scope.go:117] "RemoveContainer" containerID="df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966" Jul 10 18:32:21.795508 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:21.795492 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": container with ID starting with df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966 not found: ID does not exist" containerID="df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966" Jul 10 18:32:21.795546 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.795512 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966"} err="failed to get container status \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": rpc error: code = NotFound desc = could not find container \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": container with ID starting with df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966 not found: ID does not exist" Jul 10 18:32:21.795546 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.795525 2565 scope.go:117] "RemoveContainer" containerID="67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625" Jul 10 18:32:21.795766 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:21.795750 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": container with ID starting with 67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625 not found: ID does not exist" containerID="67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625" Jul 10 18:32:21.795831 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.795769 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625"} err="failed to get container status \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": rpc error: code = NotFound desc = could not find container \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": container with ID starting with 67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625 not found: ID does not exist" Jul 10 18:32:21.795831 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.795782 2565 scope.go:117] "RemoveContainer" containerID="a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b" Jul 10 18:32:21.796017 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:21.795999 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": container with ID starting with a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b not found: ID does not exist" containerID="a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b" Jul 10 18:32:21.796063 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.796023 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b"} err="failed to get container status \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": rpc error: code = NotFound desc = could not find container \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": container with ID starting with a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b not found: ID does not exist" Jul 10 18:32:21.796063 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.796039 2565 scope.go:117] "RemoveContainer" containerID="a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc" Jul 10 18:32:21.796369 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:21.796353 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": container with ID starting with a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc not found: ID does not exist" containerID="a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc" Jul 10 18:32:21.796448 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.796371 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc"} err="failed to get container status \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": rpc error: code = NotFound desc = could not find container \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": container with ID starting with a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc not found: ID does not exist" Jul 10 18:32:21.796448 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.796385 2565 scope.go:117] "RemoveContainer" containerID="fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550" Jul 10 18:32:21.796669 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:32:21.796643 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": container with ID starting with fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550 not found: ID does not exist" containerID="fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550" Jul 10 18:32:21.796720 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.796673 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550"} err="failed to get container status \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": rpc error: code = NotFound desc = could not find container \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": container with ID starting with fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550 not found: ID does not exist" Jul 10 18:32:21.796720 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.796686 2565 scope.go:117] "RemoveContainer" containerID="41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e" Jul 10 18:32:21.796964 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.796948 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e"} err="failed to get container status \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": rpc error: code = NotFound desc = could not find container \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": container with ID starting with 41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e not found: ID does not exist" Jul 10 18:32:21.797019 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.796966 2565 scope.go:117] "RemoveContainer" containerID="232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b" Jul 10 18:32:21.797242 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.797224 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b"} err="failed to get container status \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": rpc error: code = NotFound desc = could not find container \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": container with ID starting with 232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b not found: ID does not exist" Jul 10 18:32:21.797292 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.797244 2565 scope.go:117] "RemoveContainer" containerID="df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966" Jul 10 18:32:21.797460 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.797442 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966"} err="failed to get container status \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": rpc error: code = NotFound desc = could not find container \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": container with ID starting with df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966 not found: ID does not exist" Jul 10 18:32:21.797502 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.797459 2565 scope.go:117] "RemoveContainer" containerID="67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625" Jul 10 18:32:21.797689 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.797672 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625"} err="failed to get container status \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": rpc error: code = NotFound desc = could not find container \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": container with ID starting with 67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625 not found: ID does not exist" Jul 10 18:32:21.797689 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.797689 2565 scope.go:117] "RemoveContainer" containerID="a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b" Jul 10 18:32:21.797913 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.797893 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b"} err="failed to get container status \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": rpc error: code = NotFound desc = could not find container \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": container with ID starting with a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b not found: ID does not exist" Jul 10 18:32:21.797956 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.797914 2565 scope.go:117] "RemoveContainer" containerID="a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc" Jul 10 18:32:21.798144 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.798126 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc"} err="failed to get container status \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": rpc error: code = NotFound desc = could not find container \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": container with ID starting with a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc not found: ID does not exist" Jul 10 18:32:21.798186 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.798145 2565 scope.go:117] "RemoveContainer" containerID="fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550" Jul 10 18:32:21.798370 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.798355 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550"} err="failed to get container status \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": rpc error: code = NotFound desc = could not find container \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": container with ID starting with fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550 not found: ID does not exist" Jul 10 18:32:21.798414 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.798370 2565 scope.go:117] "RemoveContainer" containerID="41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e" Jul 10 18:32:21.798595 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.798576 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e"} err="failed to get container status \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": rpc error: code = NotFound desc = could not find container \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": container with ID starting with 41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e not found: ID does not exist" Jul 10 18:32:21.798595 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.798594 2565 scope.go:117] "RemoveContainer" containerID="232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b" Jul 10 18:32:21.798812 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.798794 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b"} err="failed to get container status \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": rpc error: code = NotFound desc = could not find container \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": container with ID starting with 232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b not found: ID does not exist" Jul 10 18:32:21.798891 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.798814 2565 scope.go:117] "RemoveContainer" containerID="df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966" Jul 10 18:32:21.799027 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.799011 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966"} err="failed to get container status \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": rpc error: code = NotFound desc = could not find container \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": container with ID starting with df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966 not found: ID does not exist" Jul 10 18:32:21.799102 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.799028 2565 scope.go:117] "RemoveContainer" containerID="67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625" Jul 10 18:32:21.799297 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.799279 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625"} err="failed to get container status \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": rpc error: code = NotFound desc = could not find container \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": container with ID starting with 67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625 not found: ID does not exist" Jul 10 18:32:21.799374 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.799298 2565 scope.go:117] "RemoveContainer" containerID="a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b" Jul 10 18:32:21.799623 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.799606 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b"} err="failed to get container status \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": rpc error: code = NotFound desc = could not find container \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": container with ID starting with a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b not found: ID does not exist" Jul 10 18:32:21.799704 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.799624 2565 scope.go:117] "RemoveContainer" containerID="a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc" Jul 10 18:32:21.799869 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.799852 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc"} err="failed to get container status \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": rpc error: code = NotFound desc = could not find container \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": container with ID starting with a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc not found: ID does not exist" Jul 10 18:32:21.799938 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.799870 2565 scope.go:117] "RemoveContainer" containerID="fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550" Jul 10 18:32:21.800244 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.800222 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550"} err="failed to get container status \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": rpc error: code = NotFound desc = could not find container \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": container with ID starting with fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550 not found: ID does not exist" Jul 10 18:32:21.800292 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.800245 2565 scope.go:117] "RemoveContainer" containerID="41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e" Jul 10 18:32:21.800475 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.800456 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e"} err="failed to get container status \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": rpc error: code = NotFound desc = could not find container \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": container with ID starting with 41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e not found: ID does not exist" Jul 10 18:32:21.800535 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.800477 2565 scope.go:117] "RemoveContainer" containerID="232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b" Jul 10 18:32:21.800689 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.800669 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b"} err="failed to get container status \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": rpc error: code = NotFound desc = could not find container \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": container with ID starting with 232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b not found: ID does not exist" Jul 10 18:32:21.800734 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.800691 2565 scope.go:117] "RemoveContainer" containerID="df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966" Jul 10 18:32:21.800919 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.800901 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966"} err="failed to get container status \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": rpc error: code = NotFound desc = could not find container \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": container with ID starting with df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966 not found: ID does not exist" Jul 10 18:32:21.800975 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.800920 2565 scope.go:117] "RemoveContainer" containerID="67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625" Jul 10 18:32:21.801135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.801113 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625"} err="failed to get container status \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": rpc error: code = NotFound desc = could not find container \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": container with ID starting with 67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625 not found: ID does not exist" Jul 10 18:32:21.801135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.801135 2565 scope.go:117] "RemoveContainer" containerID="a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b" Jul 10 18:32:21.801363 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.801344 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b"} err="failed to get container status \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": rpc error: code = NotFound desc = could not find container \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": container with ID starting with a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b not found: ID does not exist" Jul 10 18:32:21.801407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.801372 2565 scope.go:117] "RemoveContainer" containerID="a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc" Jul 10 18:32:21.801578 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.801561 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc"} err="failed to get container status \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": rpc error: code = NotFound desc = could not find container \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": container with ID starting with a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc not found: ID does not exist" Jul 10 18:32:21.801626 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.801579 2565 scope.go:117] "RemoveContainer" containerID="fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550" Jul 10 18:32:21.801786 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.801769 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550"} err="failed to get container status \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": rpc error: code = NotFound desc = could not find container \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": container with ID starting with fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550 not found: ID does not exist" Jul 10 18:32:21.801830 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.801786 2565 scope.go:117] "RemoveContainer" containerID="41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e" Jul 10 18:32:21.802006 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.801990 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e"} err="failed to get container status \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": rpc error: code = NotFound desc = could not find container \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": container with ID starting with 41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e not found: ID does not exist" Jul 10 18:32:21.802061 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.802006 2565 scope.go:117] "RemoveContainer" containerID="232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b" Jul 10 18:32:21.802244 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.802224 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b"} err="failed to get container status \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": rpc error: code = NotFound desc = could not find container \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": container with ID starting with 232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b not found: ID does not exist" Jul 10 18:32:21.802303 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.802244 2565 scope.go:117] "RemoveContainer" containerID="df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966" Jul 10 18:32:21.802457 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.802439 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966"} err="failed to get container status \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": rpc error: code = NotFound desc = could not find container \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": container with ID starting with df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966 not found: ID does not exist" Jul 10 18:32:21.802503 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.802457 2565 scope.go:117] "RemoveContainer" containerID="67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625" Jul 10 18:32:21.802680 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.802663 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625"} err="failed to get container status \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": rpc error: code = NotFound desc = could not find container \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": container with ID starting with 67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625 not found: ID does not exist" Jul 10 18:32:21.802719 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.802681 2565 scope.go:117] "RemoveContainer" containerID="a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b" Jul 10 18:32:21.802911 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.802890 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b"} err="failed to get container status \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": rpc error: code = NotFound desc = could not find container \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": container with ID starting with a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b not found: ID does not exist" Jul 10 18:32:21.802952 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.802914 2565 scope.go:117] "RemoveContainer" containerID="a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc" Jul 10 18:32:21.803225 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.803206 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc"} err="failed to get container status \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": rpc error: code = NotFound desc = could not find container \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": container with ID starting with a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc not found: ID does not exist" Jul 10 18:32:21.803225 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.803224 2565 scope.go:117] "RemoveContainer" containerID="fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550" Jul 10 18:32:21.803430 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.803415 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550"} err="failed to get container status \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": rpc error: code = NotFound desc = could not find container \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": container with ID starting with fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550 not found: ID does not exist" Jul 10 18:32:21.803469 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.803430 2565 scope.go:117] "RemoveContainer" containerID="41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e" Jul 10 18:32:21.803633 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.803618 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e"} err="failed to get container status \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": rpc error: code = NotFound desc = could not find container \"41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e\": container with ID starting with 41adf696e79ffe3207ba0284b934123de495e5a30a41805aa9de98aa9c504f6e not found: ID does not exist" Jul 10 18:32:21.803671 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.803635 2565 scope.go:117] "RemoveContainer" containerID="232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b" Jul 10 18:32:21.803830 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.803815 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b"} err="failed to get container status \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": rpc error: code = NotFound desc = could not find container \"232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b\": container with ID starting with 232e442286592fc348206d79d62007a459c996675167e0f0d60cf77cef52d27b not found: ID does not exist" Jul 10 18:32:21.803881 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.803830 2565 scope.go:117] "RemoveContainer" containerID="df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966" Jul 10 18:32:21.804021 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.804002 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966"} err="failed to get container status \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": rpc error: code = NotFound desc = could not find container \"df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966\": container with ID starting with df7c3363e0ed49822cb62bc59b292f9a5a119fb9471afce805eb5ab10398b966 not found: ID does not exist" Jul 10 18:32:21.804062 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.804020 2565 scope.go:117] "RemoveContainer" containerID="67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625" Jul 10 18:32:21.804247 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.804228 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625"} err="failed to get container status \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": rpc error: code = NotFound desc = could not find container \"67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625\": container with ID starting with 67d5e35294e4cfe6afbd9912d652704619d8910352388e2c9b94c297e0ccc625 not found: ID does not exist" Jul 10 18:32:21.804303 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.804247 2565 scope.go:117] "RemoveContainer" containerID="a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b" Jul 10 18:32:21.804476 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.804458 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b"} err="failed to get container status \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": rpc error: code = NotFound desc = could not find container \"a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b\": container with ID starting with a6b2edda34b0be38081cc7631b5017403720390704b4107ae901ea98c3579a9b not found: ID does not exist" Jul 10 18:32:21.804512 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.804477 2565 scope.go:117] "RemoveContainer" containerID="a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc" Jul 10 18:32:21.804678 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.804661 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc"} err="failed to get container status \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": rpc error: code = NotFound desc = could not find container \"a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc\": container with ID starting with a98f486a39aa4b640b3316fd5f2fed3b5b2deb8bf2ca1d35d030828ae8cee2cc not found: ID does not exist" Jul 10 18:32:21.804727 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.804678 2565 scope.go:117] "RemoveContainer" containerID="fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550" Jul 10 18:32:21.804867 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.804849 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550"} err="failed to get container status \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": rpc error: code = NotFound desc = could not find container \"fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550\": container with ID starting with fef6ec4a11f041552b386804595bafac30bc7463ad181d2ea59babebdff53550 not found: ID does not exist" Jul 10 18:32:21.809816 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.809788 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:32:21.810117 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810103 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy-web" Jul 10 18:32:21.810157 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810120 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy-web" Jul 10 18:32:21.810157 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810131 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="init-config-reloader" Jul 10 18:32:21.810157 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810137 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="init-config-reloader" Jul 10 18:32:21.810157 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810144 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy-metric" Jul 10 18:32:21.810157 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810150 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy-metric" Jul 10 18:32:21.810157 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810157 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810162 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810171 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcd36080-d958-459c-96d2-8157766352cd" containerName="console" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810176 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcd36080-d958-459c-96d2-8157766352cd" containerName="console" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810182 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="prometheus" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810187 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="prometheus" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810212 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="init-config-reloader" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810219 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="init-config-reloader" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810226 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="alertmanager" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810231 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="alertmanager" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810237 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy-web" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810242 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy-web" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810248 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="prom-label-proxy" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810253 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="prom-label-proxy" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810257 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy-thanos" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810262 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy-thanos" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810271 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="config-reloader" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810276 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="config-reloader" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810284 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="config-reloader" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810288 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="config-reloader" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810294 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810299 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810304 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="thanos-sidecar" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810309 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="thanos-sidecar" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810315 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f686b83-e37c-4b7e-9759-c6b56d56a99f" containerName="console" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810320 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f686b83-e37c-4b7e-9759-c6b56d56a99f" containerName="console" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810327 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f08196c-7a64-43e1-9dd4-bf32a34b2844" containerName="registry" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810332 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f08196c-7a64-43e1-9dd4-bf32a34b2844" containerName="registry" Jul 10 18:32:21.810372 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810382 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f686b83-e37c-4b7e-9759-c6b56d56a99f" containerName="console" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810390 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810397 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy-thanos" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810402 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="config-reloader" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810411 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f08196c-7a64-43e1-9dd4-bf32a34b2844" containerName="registry" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810416 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="alertmanager" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810421 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810426 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy-metric" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810433 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="thanos-sidecar" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810441 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="kube-rbac-proxy-web" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810448 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="prometheus" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810453 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" containerName="kube-rbac-proxy-web" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810458 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcd36080-d958-459c-96d2-8157766352cd" containerName="console" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810463 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="config-reloader" Jul 10 18:32:21.811106 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.810468 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="de6c1f0e-0c3c-49fe-adb4-a000a85ee516" containerName="prom-label-proxy" Jul 10 18:32:21.815943 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.815926 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.820858 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.820835 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 10 18:32:21.820995 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.820922 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 10 18:32:21.820995 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.820981 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-23sdhkjol8crs\"" Jul 10 18:32:21.821120 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.821025 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 10 18:32:21.821241 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.821226 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 10 18:32:21.821854 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.821839 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 10 18:32:21.821913 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.821886 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 10 18:32:21.823908 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.823886 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 10 18:32:21.823908 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.823897 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 10 18:32:21.824065 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.823916 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 10 18:32:21.824456 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.824336 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 10 18:32:21.824814 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.824739 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 10 18:32:21.825124 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.825089 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 10 18:32:21.825124 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.825117 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-l5ms7\"" Jul 10 18:32:21.826387 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.826370 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 10 18:32:21.829731 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.829712 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 10 18:32:21.833051 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.833028 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:32:21.931115 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931073 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931305 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931120 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931305 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931171 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931305 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931255 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931305 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931275 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931305 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931291 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b11dab4-de81-4582-8c66-6584ce5e5417-config-out\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931305 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931307 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931369 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b11dab4-de81-4582-8c66-6584ce5e5417-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931411 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knbrb\" (UniqueName: \"kubernetes.io/projected/6b11dab4-de81-4582-8c66-6584ce5e5417-kube-api-access-knbrb\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931438 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-config\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931473 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-web-config\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931504 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931493 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931644 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931525 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931644 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931555 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931644 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931569 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931644 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931591 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931644 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931627 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931782 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931670 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931782 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931690 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:21.931782 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:21.931709 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.032563 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032483 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.032563 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032520 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.032563 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032540 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.032756 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032617 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.032756 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032650 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.032756 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032679 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.032756 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032702 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.032756 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032736 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033007 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032788 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033007 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032819 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033007 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032843 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b11dab4-de81-4582-8c66-6584ce5e5417-config-out\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033007 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032866 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033007 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032894 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b11dab4-de81-4582-8c66-6584ce5e5417-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033007 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032934 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knbrb\" (UniqueName: \"kubernetes.io/projected/6b11dab4-de81-4582-8c66-6584ce5e5417-kube-api-access-knbrb\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033007 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.032972 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-config\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033007 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.033001 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-web-config\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033564 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.033027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033564 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.033063 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033564 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.033094 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033564 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.033117 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033564 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.033186 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033564 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.033427 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.033768 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.033658 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.034060 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.034032 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.034875 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.034848 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.034983 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.034891 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.036273 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.035901 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.036273 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.036218 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b11dab4-de81-4582-8c66-6584ce5e5417-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.036426 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.036286 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-web-config\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.036790 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.036765 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.036923 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.036874 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.037015 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.036973 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6b11dab4-de81-4582-8c66-6584ce5e5417-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.037636 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.037590 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.037732 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.037707 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.037811 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.037788 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b11dab4-de81-4582-8c66-6584ce5e5417-config-out\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.038593 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.038566 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.038720 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.038701 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-config\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.039415 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.039393 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.039490 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.039424 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6b11dab4-de81-4582-8c66-6584ce5e5417-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.045784 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.045761 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knbrb\" (UniqueName: \"kubernetes.io/projected/6b11dab4-de81-4582-8c66-6584ce5e5417-kube-api-access-knbrb\") pod \"prometheus-k8s-0\" (UID: \"6b11dab4-de81-4582-8c66-6584ce5e5417\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.127789 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.127751 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:22.291033 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.290950 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:32:22.294775 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:32:22.294742 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b11dab4_de81_4582_8c66_6584ce5e5417.slice/crio-ae8e0cf6fcfb0349685e7e9c09998879a2f84fd3ba630f1b57025b6b8015985a WatchSource:0}: Error finding container ae8e0cf6fcfb0349685e7e9c09998879a2f84fd3ba630f1b57025b6b8015985a: Status 404 returned error can't find the container with id ae8e0cf6fcfb0349685e7e9c09998879a2f84fd3ba630f1b57025b6b8015985a Jul 10 18:32:22.745833 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.745798 2565 generic.go:356] "Generic (PLEG): container finished" podID="6b11dab4-de81-4582-8c66-6584ce5e5417" containerID="b8d438a2ba0924ffcadcaf5ba0f2383495bba4e87b73e018094c921ae313199e" exitCode=0 Jul 10 18:32:22.746428 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.745885 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b11dab4-de81-4582-8c66-6584ce5e5417","Type":"ContainerDied","Data":"b8d438a2ba0924ffcadcaf5ba0f2383495bba4e87b73e018094c921ae313199e"} Jul 10 18:32:22.746428 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:22.745963 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b11dab4-de81-4582-8c66-6584ce5e5417","Type":"ContainerStarted","Data":"ae8e0cf6fcfb0349685e7e9c09998879a2f84fd3ba630f1b57025b6b8015985a"} Jul 10 18:32:23.295257 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:23.295170 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a91daf7-9723-454e-bd23-baabdfbcd14a" path="/var/lib/kubelet/pods/5a91daf7-9723-454e-bd23-baabdfbcd14a/volumes" Jul 10 18:32:23.755545 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:23.755515 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b11dab4-de81-4582-8c66-6584ce5e5417","Type":"ContainerStarted","Data":"3f07a8f35f013caaaf0b07aa444af3ae0fc6827e5be75c3237fb70c69bd77874"} Jul 10 18:32:23.755911 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:23.755575 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b11dab4-de81-4582-8c66-6584ce5e5417","Type":"ContainerStarted","Data":"0b9796b3598ac16402a092c41f805fca974676e99d3b5ca93e9b6371247079bc"} Jul 10 18:32:23.755911 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:23.755602 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b11dab4-de81-4582-8c66-6584ce5e5417","Type":"ContainerStarted","Data":"c88e54f9169b1fe99820f153e3bed72c6c64f58c8044baf95638f6f976d2e067"} Jul 10 18:32:23.755911 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:23.755629 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b11dab4-de81-4582-8c66-6584ce5e5417","Type":"ContainerStarted","Data":"f044111d5a4ab8fa8053e5e55c0154cf857ad92563b2f3f9e5cf53527e100203"} Jul 10 18:32:23.755911 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:23.755652 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b11dab4-de81-4582-8c66-6584ce5e5417","Type":"ContainerStarted","Data":"3a7187c359779d278f3fafeeafd2db97559e8c216e251c5174aa242e031c29c8"} Jul 10 18:32:23.755911 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:23.755679 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b11dab4-de81-4582-8c66-6584ce5e5417","Type":"ContainerStarted","Data":"556a4b84c59c0b31cb825a602ace610f9cc7105f829db8c12c5198403cb58ac5"} Jul 10 18:32:23.791932 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:23.791879 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.77334373 podStartE2EDuration="2.791860811s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.517081ms" isStatefulPod=false podCreationTimestamp="2026-07-10 18:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:32:23.790465053 +0000 UTC m=+388.996305165" watchObservedRunningTime="2026-07-10 18:32:23.791860811 +0000 UTC m=+388.997700866" Jul 10 18:32:23.792592 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:23.792562 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:23.857363 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:23.857320 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:27.128817 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:27.128778 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:32:28.806048 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:28.806000 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:28.871588 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:28.871551 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:34.089041 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:34.089006 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:34.187936 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:34.187903 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:39.707975 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:39.707936 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74b7d5dddf-dzqgn"] Jul 10 18:32:39.882754 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:39.882720 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:39.982818 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:39.982723 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:40.054342 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:40.054302 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:58.046414 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:32:58.046370 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:04.727590 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:04.727529 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-74b7d5dddf-dzqgn" podUID="996c457d-48e9-4045-bded-270d270dd4e0" containerName="console" containerID="cri-o://bdcdb0c93bc32ccda377cd9ea99cdda343af9d454e52a24ed0c2a84f7506cf8d" gracePeriod=15 Jul 10 18:33:04.937043 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:04.937005 2565 generic.go:356] "Generic (PLEG): container finished" podID="996c457d-48e9-4045-bded-270d270dd4e0" containerID="bdcdb0c93bc32ccda377cd9ea99cdda343af9d454e52a24ed0c2a84f7506cf8d" exitCode=0 Jul 10 18:33:04.937224 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:04.937071 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b7d5dddf-dzqgn" event={"ID":"996c457d-48e9-4045-bded-270d270dd4e0","Type":"ContainerDied","Data":"bdcdb0c93bc32ccda377cd9ea99cdda343af9d454e52a24ed0c2a84f7506cf8d"} Jul 10 18:33:04.976877 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:04.976849 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:33:05.120454 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.120419 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-oauth-serving-cert\") pod \"996c457d-48e9-4045-bded-270d270dd4e0\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " Jul 10 18:33:05.120650 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.120497 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-serving-cert\") pod \"996c457d-48e9-4045-bded-270d270dd4e0\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " Jul 10 18:33:05.120650 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.120521 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzsn9\" (UniqueName: \"kubernetes.io/projected/996c457d-48e9-4045-bded-270d270dd4e0-kube-api-access-rzsn9\") pod \"996c457d-48e9-4045-bded-270d270dd4e0\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " Jul 10 18:33:05.120650 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.120547 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-console-config\") pod \"996c457d-48e9-4045-bded-270d270dd4e0\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " Jul 10 18:33:05.120650 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.120589 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-trusted-ca-bundle\") pod \"996c457d-48e9-4045-bded-270d270dd4e0\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " Jul 10 18:33:05.120893 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.120685 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-service-ca\") pod \"996c457d-48e9-4045-bded-270d270dd4e0\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " Jul 10 18:33:05.120893 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.120790 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-oauth-config\") pod \"996c457d-48e9-4045-bded-270d270dd4e0\" (UID: \"996c457d-48e9-4045-bded-270d270dd4e0\") " Jul 10 18:33:05.120893 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.120850 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "996c457d-48e9-4045-bded-270d270dd4e0" (UID: "996c457d-48e9-4045-bded-270d270dd4e0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:33:05.121053 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.120986 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "996c457d-48e9-4045-bded-270d270dd4e0" (UID: "996c457d-48e9-4045-bded-270d270dd4e0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:33:05.121110 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.121064 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-console-config" (OuterVolumeSpecName: "console-config") pod "996c457d-48e9-4045-bded-270d270dd4e0" (UID: "996c457d-48e9-4045-bded-270d270dd4e0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:33:05.121373 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.121353 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-console-config\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:33:05.121461 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.121386 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-trusted-ca-bundle\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:33:05.121461 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.121402 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-oauth-serving-cert\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:33:05.121461 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.121349 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-service-ca" (OuterVolumeSpecName: "service-ca") pod "996c457d-48e9-4045-bded-270d270dd4e0" (UID: "996c457d-48e9-4045-bded-270d270dd4e0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:33:05.122900 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.122879 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "996c457d-48e9-4045-bded-270d270dd4e0" (UID: "996c457d-48e9-4045-bded-270d270dd4e0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:33:05.122900 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.122883 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/996c457d-48e9-4045-bded-270d270dd4e0-kube-api-access-rzsn9" (OuterVolumeSpecName: "kube-api-access-rzsn9") pod "996c457d-48e9-4045-bded-270d270dd4e0" (UID: "996c457d-48e9-4045-bded-270d270dd4e0"). InnerVolumeSpecName "kube-api-access-rzsn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:33:05.123082 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.123049 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "996c457d-48e9-4045-bded-270d270dd4e0" (UID: "996c457d-48e9-4045-bded-270d270dd4e0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:33:05.222727 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.222686 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/996c457d-48e9-4045-bded-270d270dd4e0-service-ca\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:33:05.222727 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.222717 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-oauth-config\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:33:05.222727 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.222727 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/996c457d-48e9-4045-bded-270d270dd4e0-console-serving-cert\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:33:05.222955 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.222739 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzsn9\" (UniqueName: \"kubernetes.io/projected/996c457d-48e9-4045-bded-270d270dd4e0-kube-api-access-rzsn9\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:33:05.947999 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.947967 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b7d5dddf-dzqgn" event={"ID":"996c457d-48e9-4045-bded-270d270dd4e0","Type":"ContainerDied","Data":"fd4d0ac8fa0123613f63f1e1828207f937ab96e6688958123ab6db9aeeae6ee3"} Jul 10 18:33:05.948473 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.948058 2565 scope.go:117] "RemoveContainer" containerID="bdcdb0c93bc32ccda377cd9ea99cdda343af9d454e52a24ed0c2a84f7506cf8d" Jul 10 18:33:05.948473 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.947977 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b7d5dddf-dzqgn" Jul 10 18:33:05.966623 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.966597 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74b7d5dddf-dzqgn"] Jul 10 18:33:05.971565 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:05.971542 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74b7d5dddf-dzqgn"] Jul 10 18:33:07.294158 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:07.294126 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="996c457d-48e9-4045-bded-270d270dd4e0" path="/var/lib/kubelet/pods/996c457d-48e9-4045-bded-270d270dd4e0/volumes" Jul 10 18:33:22.128214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:22.128098 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:33:22.143889 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:22.143858 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:33:23.031437 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:23.031398 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:33:28.698983 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:28.698947 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:28.807918 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:28.807889 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:50.584400 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:50.584359 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:51.995588 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:51.995546 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:54.193668 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:54.193580 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:54.293481 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:54.293445 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:55.839597 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:55.839554 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:55.937601 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:55.937571 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:56.614440 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:56.614405 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:56.717147 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:56.717111 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:58.223181 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:58.223139 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:58.440228 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:33:58.440149 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:01.216217 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:01.216165 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:01.278227 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:01.278158 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:03.989321 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:03.989264 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:04.225388 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.225351 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dfllt"] Jul 10 18:34:04.225842 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.225824 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="996c457d-48e9-4045-bded-270d270dd4e0" containerName="console" Jul 10 18:34:04.225842 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.225843 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="996c457d-48e9-4045-bded-270d270dd4e0" containerName="console" Jul 10 18:34:04.225966 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.225917 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="996c457d-48e9-4045-bded-270d270dd4e0" containerName="console" Jul 10 18:34:04.230229 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.230184 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" Jul 10 18:34:04.233649 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.233614 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-p892k\"" Jul 10 18:34:04.233789 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.233616 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:04.234962 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.234942 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 18:34:04.235243 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.234968 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 18:34:04.248107 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.248039 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dfllt"] Jul 10 18:34:04.335587 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.335546 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbn9l\" (UniqueName: \"kubernetes.io/projected/6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2-kube-api-access-nbn9l\") pod \"cert-manager-webhook-6486ffd997-dfllt\" (UID: \"6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" Jul 10 18:34:04.335764 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.335597 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dfllt\" (UID: \"6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" Jul 10 18:34:04.436694 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.436655 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbn9l\" (UniqueName: \"kubernetes.io/projected/6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2-kube-api-access-nbn9l\") pod \"cert-manager-webhook-6486ffd997-dfllt\" (UID: \"6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" Jul 10 18:34:04.436882 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.436709 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dfllt\" (UID: \"6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" Jul 10 18:34:04.446599 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.446560 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dfllt\" (UID: \"6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" Jul 10 18:34:04.446766 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.446631 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbn9l\" (UniqueName: \"kubernetes.io/projected/6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2-kube-api-access-nbn9l\") pod \"cert-manager-webhook-6486ffd997-dfllt\" (UID: \"6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" Jul 10 18:34:04.549124 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.549026 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" Jul 10 18:34:04.681538 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:04.681514 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dfllt"] Jul 10 18:34:04.684264 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:34:04.684233 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c6683a1_cd75_4bb5_bc23_02b9a5bacfe2.slice/crio-20f587f68885125588e2e6c7311fe514e831867b14533dae1fdd92d113fd9f93 WatchSource:0}: Error finding container 20f587f68885125588e2e6c7311fe514e831867b14533dae1fdd92d113fd9f93: Status 404 returned error can't find the container with id 20f587f68885125588e2e6c7311fe514e831867b14533dae1fdd92d113fd9f93 Jul 10 18:34:05.186043 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:05.185995 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" event={"ID":"6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2","Type":"ContainerStarted","Data":"20f587f68885125588e2e6c7311fe514e831867b14533dae1fdd92d113fd9f93"} Jul 10 18:34:05.441031 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:05.440936 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:08.195500 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:08.195461 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:08.199825 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:08.199800 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" event={"ID":"6c6683a1-cd75-4bb5-bc23-02b9a5bacfe2","Type":"ContainerStarted","Data":"361808820b0caf86ab69be94ed5f343f55ae732b154c7b8f409b73612fcff8e3"} Jul 10 18:34:08.199967 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:08.199922 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" Jul 10 18:34:08.230977 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:08.230916 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" podStartSLOduration=1.380138924 podStartE2EDuration="4.230896744s" totalImagesPullingTime="2.85075782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:34:08.229107216 +0000 UTC m=+493.434947272" watchObservedRunningTime="2026-07-10 18:34:08.230896744 +0000 UTC m=+493.436736800" Jul 10 18:34:08.295559 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:08.295515 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:14.206254 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:14.206222 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dfllt" Jul 10 18:34:14.707921 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:14.707885 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:14.812632 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:14.812591 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:16.952758 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:16.952723 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:23.375669 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:23.375630 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:23.436603 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:23.436562 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:29.599121 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:29.599075 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:29.701742 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:29.701694 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:30.210995 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.210952 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:30.582725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.582640 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8"] Jul 10 18:34:30.590240 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.590214 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.593865 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.593836 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 18:34:30.594006 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.593849 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 18:34:30.594006 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.593849 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 18:34:30.594222 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.594185 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fkw6b\"" Jul 10 18:34:30.594334 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.594226 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 18:34:30.605552 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.605521 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8"] Jul 10 18:34:30.630319 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.630293 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e6e67385-387b-4b0f-944b-d8efa2f2c13c-manifests\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.630556 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.630349 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6e67385-387b-4b0f-944b-d8efa2f2c13c-tmp\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.630556 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.630381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6e67385-387b-4b0f-944b-d8efa2f2c13c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.630556 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.630451 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hldwk\" (UniqueName: \"kubernetes.io/projected/e6e67385-387b-4b0f-944b-d8efa2f2c13c-kube-api-access-hldwk\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.630556 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.630519 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e6e67385-387b-4b0f-944b-d8efa2f2c13c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.731599 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.731562 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6e67385-387b-4b0f-944b-d8efa2f2c13c-tmp\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.731599 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.731607 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6e67385-387b-4b0f-944b-d8efa2f2c13c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.731824 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.731643 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hldwk\" (UniqueName: \"kubernetes.io/projected/e6e67385-387b-4b0f-944b-d8efa2f2c13c-kube-api-access-hldwk\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.731824 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.731673 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e6e67385-387b-4b0f-944b-d8efa2f2c13c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.731824 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.731717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e6e67385-387b-4b0f-944b-d8efa2f2c13c-manifests\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.732076 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.732052 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e6e67385-387b-4b0f-944b-d8efa2f2c13c-manifests\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.734012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.733977 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6e67385-387b-4b0f-944b-d8efa2f2c13c-tmp\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.734225 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.734181 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e6e67385-387b-4b0f-944b-d8efa2f2c13c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.734298 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.734263 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e6e67385-387b-4b0f-944b-d8efa2f2c13c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.744866 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.744836 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hldwk\" (UniqueName: \"kubernetes.io/projected/e6e67385-387b-4b0f-944b-d8efa2f2c13c-kube-api-access-hldwk\") pod \"opendatahub-operator-controller-manager-5488d6dcc-wvmq8\" (UID: \"e6e67385-387b-4b0f-944b-d8efa2f2c13c\") " pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.796933 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.796892 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:30.901389 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.901349 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:30.926695 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:30.926649 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:31.028476 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:31.028432 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:31.081019 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:31.080994 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8"] Jul 10 18:34:31.083437 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:34:31.083409 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e67385_387b_4b0f_944b_d8efa2f2c13c.slice/crio-84b350f8dd229962b96a793c90c2d681aa521a9c44efcdf2092bcbc5df9f43f1 WatchSource:0}: Error finding container 84b350f8dd229962b96a793c90c2d681aa521a9c44efcdf2092bcbc5df9f43f1: Status 404 returned error can't find the container with id 84b350f8dd229962b96a793c90c2d681aa521a9c44efcdf2092bcbc5df9f43f1 Jul 10 18:34:31.121371 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:31.121330 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:31.225685 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:31.225589 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:31.295619 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:31.295584 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" event={"ID":"e6e67385-387b-4b0f-944b-d8efa2f2c13c","Type":"ContainerStarted","Data":"84b350f8dd229962b96a793c90c2d681aa521a9c44efcdf2092bcbc5df9f43f1"} Jul 10 18:34:32.180330 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:32.180286 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:34.316942 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:34.316841 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" event={"ID":"e6e67385-387b-4b0f-944b-d8efa2f2c13c","Type":"ContainerStarted","Data":"3dc94235d310c22ecb1889787c554db1ed4c35e1f4f2ee44dea7cdff64651e66"} Jul 10 18:34:35.322815 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:35.322778 2565 generic.go:356] "Generic (PLEG): container finished" podID="e6e67385-387b-4b0f-944b-d8efa2f2c13c" containerID="3dc94235d310c22ecb1889787c554db1ed4c35e1f4f2ee44dea7cdff64651e66" exitCode=0 Jul 10 18:34:35.323235 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:35.322866 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" event={"ID":"e6e67385-387b-4b0f-944b-d8efa2f2c13c","Type":"ContainerDied","Data":"3dc94235d310c22ecb1889787c554db1ed4c35e1f4f2ee44dea7cdff64651e66"} Jul 10 18:34:36.329333 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:36.329294 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" event={"ID":"e6e67385-387b-4b0f-944b-d8efa2f2c13c","Type":"ContainerStarted","Data":"9516a5921512344c619f4be9d104cf190cc79e16a87f8c5cfda13e27a8dc301c"} Jul 10 18:34:36.329713 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:36.329408 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:36.356563 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:36.356513 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" podStartSLOduration=2.055791807 podStartE2EDuration="6.356494344s" totalImagesPullingTime="3.213056359s" totalInitContainerRuntime="1.087646178s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:34:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:34:36.354673291 +0000 UTC m=+521.560513361" watchObservedRunningTime="2026-07-10 18:34:36.356494344 +0000 UTC m=+521.562334399" Jul 10 18:34:38.279879 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:38.279841 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:38.589615 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:38.589519 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:41.822436 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:41.822396 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:41.923249 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:41.923216 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:42.125206 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:42.125156 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:43.204390 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:43.204353 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:43.305073 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:43.305039 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:45.900303 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:45.900267 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:47.336092 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:47.336061 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5488d6dcc-wvmq8" Jul 10 18:34:48.556970 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:48.556931 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:49.424461 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.424427 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d"] Jul 10 18:34:49.433121 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.433093 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.436966 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.436940 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 10 18:34:49.436966 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.436953 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 18:34:49.438512 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.438490 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-57znh\"" Jul 10 18:34:49.438648 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.438541 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 10 18:34:49.438648 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.438557 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 10 18:34:49.438648 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.438503 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:34:49.439120 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.439102 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d"] Jul 10 18:34:49.490247 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.490213 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-metrics-cert\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.490425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.490254 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd5qb\" (UniqueName: \"kubernetes.io/projected/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-kube-api-access-cd5qb\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.490425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.490331 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-manager-config\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.490425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.490392 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-cert\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.591728 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.591679 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-cert\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.592131 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.591755 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-metrics-cert\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.592131 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.591780 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd5qb\" (UniqueName: \"kubernetes.io/projected/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-kube-api-access-cd5qb\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.592131 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.591818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-manager-config\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.592644 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.592621 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-manager-config\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.594365 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.594342 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-cert\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.594546 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.594477 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-metrics-cert\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.601388 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.601364 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd5qb\" (UniqueName: \"kubernetes.io/projected/5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f-kube-api-access-cd5qb\") pod \"lws-controller-manager-67d57b55c8-bnm5d\" (UID: \"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f\") " pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.744334 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.744240 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:49.896626 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.896593 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d"] Jul 10 18:34:49.899940 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:34:49.899899 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5771e988_ea9a_4da1_89fa_4bbf0bb5ac7f.slice/crio-ded713854c6eaa893e5ddb68615f6f5ed5a9face6261f106a8e0eba563db0338 WatchSource:0}: Error finding container ded713854c6eaa893e5ddb68615f6f5ed5a9face6261f106a8e0eba563db0338: Status 404 returned error can't find the container with id ded713854c6eaa893e5ddb68615f6f5ed5a9face6261f106a8e0eba563db0338 Jul 10 18:34:49.985632 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:49.985600 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:50.123979 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:50.123941 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:50.395769 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:50.395642 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" event={"ID":"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f","Type":"ContainerStarted","Data":"ded713854c6eaa893e5ddb68615f6f5ed5a9face6261f106a8e0eba563db0338"} Jul 10 18:34:50.863478 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:50.863429 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:50.977584 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:50.977496 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:51.234787 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:51.234676 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:51.523695 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:51.523550 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:51.775873 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:51.775785 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:52.153162 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.153129 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:52.266521 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.266383 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:52.274790 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.274164 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx"] Jul 10 18:34:52.278444 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.278419 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:52.285912 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.285882 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 18:34:52.286060 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.285975 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 18:34:52.286133 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.285882 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 18:34:52.288811 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.288373 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-q8vp2\"" Jul 10 18:34:52.288811 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.288498 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 18:34:52.294812 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.294750 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx"] Jul 10 18:34:52.417256 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.417106 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-tls-certs\") pod \"kube-auth-proxy-5dd47df98c-9fxzx\" (UID: \"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5\") " pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:52.417256 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.417157 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-tmp\") pod \"kube-auth-proxy-5dd47df98c-9fxzx\" (UID: \"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5\") " pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:52.417498 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.417389 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mlzw\" (UniqueName: \"kubernetes.io/projected/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-kube-api-access-4mlzw\") pod \"kube-auth-proxy-5dd47df98c-9fxzx\" (UID: \"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5\") " pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:52.518507 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.518471 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mlzw\" (UniqueName: \"kubernetes.io/projected/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-kube-api-access-4mlzw\") pod \"kube-auth-proxy-5dd47df98c-9fxzx\" (UID: \"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5\") " pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:52.518688 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.518522 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-tls-certs\") pod \"kube-auth-proxy-5dd47df98c-9fxzx\" (UID: \"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5\") " pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:52.518688 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.518542 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-tmp\") pod \"kube-auth-proxy-5dd47df98c-9fxzx\" (UID: \"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5\") " pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:52.518688 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:34:52.518634 2565 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 10 18:34:52.518860 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:34:52.518700 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-tls-certs podName:c26b438b-cb32-4cd7-b3d0-c1b11b483bd5 nodeName:}" failed. No retries permitted until 2026-07-10 18:34:53.018683088 +0000 UTC m=+538.224523120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-tls-certs") pod "kube-auth-proxy-5dd47df98c-9fxzx" (UID: "c26b438b-cb32-4cd7-b3d0-c1b11b483bd5") : secret "kube-auth-proxy-tls" not found Jul 10 18:34:52.520723 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.520704 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-tmp\") pod \"kube-auth-proxy-5dd47df98c-9fxzx\" (UID: \"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5\") " pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:52.528586 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.528562 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mlzw\" (UniqueName: \"kubernetes.io/projected/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-kube-api-access-4mlzw\") pod \"kube-auth-proxy-5dd47df98c-9fxzx\" (UID: \"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5\") " pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:52.584943 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:52.584900 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:53.023355 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:53.023317 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-tls-certs\") pod \"kube-auth-proxy-5dd47df98c-9fxzx\" (UID: \"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5\") " pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:53.025614 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:53.025589 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c26b438b-cb32-4cd7-b3d0-c1b11b483bd5-tls-certs\") pod \"kube-auth-proxy-5dd47df98c-9fxzx\" (UID: \"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5\") " pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:53.198150 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:53.198115 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" Jul 10 18:34:53.349587 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:53.349560 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx"] Jul 10 18:34:53.352173 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:34:53.352145 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc26b438b_cb32_4cd7_b3d0_c1b11b483bd5.slice/crio-fff87e54191b3a8dec6a3aa0dc2e7ae89273bdf975cfb8e2037f36ce9bd5fad9 WatchSource:0}: Error finding container fff87e54191b3a8dec6a3aa0dc2e7ae89273bdf975cfb8e2037f36ce9bd5fad9: Status 404 returned error can't find the container with id fff87e54191b3a8dec6a3aa0dc2e7ae89273bdf975cfb8e2037f36ce9bd5fad9 Jul 10 18:34:53.414662 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:53.414626 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" event={"ID":"5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f","Type":"ContainerStarted","Data":"d8bf34ba178fc02272d558798fe5264d9f269208ee50f30b0d2250fcd738fdcc"} Jul 10 18:34:53.414883 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:53.414742 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:34:53.416105 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:53.416072 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" event={"ID":"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5","Type":"ContainerStarted","Data":"fff87e54191b3a8dec6a3aa0dc2e7ae89273bdf975cfb8e2037f36ce9bd5fad9"} Jul 10 18:34:53.450158 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:53.450101 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" podStartSLOduration=1.7668111020000001 podStartE2EDuration="4.450083043s" totalImagesPullingTime="2.683271941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:34:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:34:53.448773626 +0000 UTC m=+538.654613705" watchObservedRunningTime="2026-07-10 18:34:53.450083043 +0000 UTC m=+538.655923099" Jul 10 18:34:53.538103 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:53.538069 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:53.615552 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:53.615519 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:54.534471 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:54.534426 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:56.347130 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:56.347106 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 18:34:56.432977 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:56.432947 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" event={"ID":"c26b438b-cb32-4cd7-b3d0-c1b11b483bd5","Type":"ContainerStarted","Data":"3c52d3fc9d23a4fb6e559b7b62c68c7633fddef81f8f56b842011a7c95225d5d"} Jul 10 18:34:57.457796 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:34:57.457742 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5dd47df98c-9fxzx" podStartSLOduration=2.467878944 podStartE2EDuration="5.457724232s" totalImagesPullingTime="2.989845288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:34:57.456596366 +0000 UTC m=+542.662436443" watchObservedRunningTime="2026-07-10 18:34:57.457724232 +0000 UTC m=+542.663564286" Jul 10 18:35:00.871581 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:00.871540 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:00.988123 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:00.988082 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:01.111954 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:01.111913 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:01.851994 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:01.851964 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:02.514816 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:02.514781 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:02.617652 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:02.617599 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:03.216657 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:03.216620 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:03.311719 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:03.311676 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:03.821258 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:03.821227 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:04.424886 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:04.424852 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-67d57b55c8-bnm5d" Jul 10 18:35:06.799751 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:06.799716 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:51.453632 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:51.453587 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:52.642887 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:52.642845 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:55.190571 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:55.190538 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:35:55.192330 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:35:55.192310 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:36:14.744253 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:14.744212 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:14.843690 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:14.843651 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:14.948615 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:14.948576 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:22.424970 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:22.424886 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:22.685592 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:22.685505 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:25.806721 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:25.806683 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:26.030139 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:26.030096 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:26.127630 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:26.127600 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:27.571915 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:27.571876 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:29.902906 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:29.902869 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc"] Jul 10 18:36:29.906358 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:29.906341 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc" Jul 10 18:36:29.909690 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:29.909671 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 10 18:36:29.909801 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:29.909705 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 18:36:29.911116 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:29.911098 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-l69j2\"" Jul 10 18:36:29.911231 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:29.911142 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 18:36:29.917456 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:29.917435 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc"] Jul 10 18:36:30.005347 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:30.005314 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6tkr\" (UniqueName: \"kubernetes.io/projected/31e94cf1-2771-4f25-b24b-a54fa25b1c1f-kube-api-access-s6tkr\") pod \"dns-operator-controller-manager-864dd78c4c-cl7kc\" (UID: \"31e94cf1-2771-4f25-b24b-a54fa25b1c1f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc" Jul 10 18:36:30.106419 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:30.106381 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6tkr\" (UniqueName: \"kubernetes.io/projected/31e94cf1-2771-4f25-b24b-a54fa25b1c1f-kube-api-access-s6tkr\") pod \"dns-operator-controller-manager-864dd78c4c-cl7kc\" (UID: \"31e94cf1-2771-4f25-b24b-a54fa25b1c1f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc" Jul 10 18:36:30.126971 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:30.126934 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6tkr\" (UniqueName: \"kubernetes.io/projected/31e94cf1-2771-4f25-b24b-a54fa25b1c1f-kube-api-access-s6tkr\") pod \"dns-operator-controller-manager-864dd78c4c-cl7kc\" (UID: \"31e94cf1-2771-4f25-b24b-a54fa25b1c1f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc" Jul 10 18:36:30.217412 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:30.217322 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc" Jul 10 18:36:30.356276 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:30.356250 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc"] Jul 10 18:36:30.358384 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:36:30.358347 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e94cf1_2771_4f25_b24b_a54fa25b1c1f.slice/crio-81f485f248017fc72cb7c60ad6f71b0a0e94484107a805de84389090c103ce81 WatchSource:0}: Error finding container 81f485f248017fc72cb7c60ad6f71b0a0e94484107a805de84389090c103ce81: Status 404 returned error can't find the container with id 81f485f248017fc72cb7c60ad6f71b0a0e94484107a805de84389090c103ce81 Jul 10 18:36:30.360388 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:30.360369 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:36:30.856736 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:30.856700 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc" event={"ID":"31e94cf1-2771-4f25-b24b-a54fa25b1c1f","Type":"ContainerStarted","Data":"81f485f248017fc72cb7c60ad6f71b0a0e94484107a805de84389090c103ce81"} Jul 10 18:36:30.880150 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:30.880115 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:32.870840 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:32.870787 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc" event={"ID":"31e94cf1-2771-4f25-b24b-a54fa25b1c1f","Type":"ContainerStarted","Data":"cd62e1166bc8a2510516b8bdcc02206bb83236ecd8db13642625f3ad5c6d9c72"} Jul 10 18:36:32.871406 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:32.870869 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc" Jul 10 18:36:32.874095 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:32.874069 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds"] Jul 10 18:36:32.877437 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:32.877414 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:32.880059 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:32.880037 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jbvrj\"" Jul 10 18:36:32.894664 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:32.894591 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds"] Jul 10 18:36:32.894948 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:32.894908 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc" podStartSLOduration=1.612322239 podStartE2EDuration="3.894895317s" totalImagesPullingTime="2.282573078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:36:32.891664389 +0000 UTC m=+638.097504445" watchObservedRunningTime="2026-07-10 18:36:32.894895317 +0000 UTC m=+638.100735372" Jul 10 18:36:33.002956 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:33.002920 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:33.032545 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:33.032515 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9xkt\" (UniqueName: \"kubernetes.io/projected/13d80b15-0300-4a9e-a59d-9b3808657862-kube-api-access-l9xkt\") pod \"limitador-operator-controller-manager-859d7b78bd-wxnds\" (UID: \"13d80b15-0300-4a9e-a59d-9b3808657862\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:33.133171 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:33.133127 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9xkt\" (UniqueName: \"kubernetes.io/projected/13d80b15-0300-4a9e-a59d-9b3808657862-kube-api-access-l9xkt\") pod \"limitador-operator-controller-manager-859d7b78bd-wxnds\" (UID: \"13d80b15-0300-4a9e-a59d-9b3808657862\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:33.150736 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:33.150670 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9xkt\" (UniqueName: \"kubernetes.io/projected/13d80b15-0300-4a9e-a59d-9b3808657862-kube-api-access-l9xkt\") pod \"limitador-operator-controller-manager-859d7b78bd-wxnds\" (UID: \"13d80b15-0300-4a9e-a59d-9b3808657862\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:33.187748 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:33.187713 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:33.249568 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:33.249537 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:33.324324 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:33.324300 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds"] Jul 10 18:36:33.326577 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:36:33.326547 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d80b15_0300_4a9e_a59d_9b3808657862.slice/crio-1b551366fdc8379f8f540eb6dfe9e280644658d83ef689a1e70066b23a8bd84a WatchSource:0}: Error finding container 1b551366fdc8379f8f540eb6dfe9e280644658d83ef689a1e70066b23a8bd84a: Status 404 returned error can't find the container with id 1b551366fdc8379f8f540eb6dfe9e280644658d83ef689a1e70066b23a8bd84a Jul 10 18:36:33.877034 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:33.876997 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" event={"ID":"13d80b15-0300-4a9e-a59d-9b3808657862","Type":"ContainerStarted","Data":"1b551366fdc8379f8f540eb6dfe9e280644658d83ef689a1e70066b23a8bd84a"} Jul 10 18:36:37.898672 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:37.898631 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" event={"ID":"13d80b15-0300-4a9e-a59d-9b3808657862","Type":"ContainerStarted","Data":"7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600"} Jul 10 18:36:37.899069 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:37.898783 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:37.916265 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:37.916210 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" podStartSLOduration=2.391828268 podStartE2EDuration="5.916173569s" totalImagesPullingTime="3.524345301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:36:37.915067125 +0000 UTC m=+643.120907184" watchObservedRunningTime="2026-07-10 18:36:37.916173569 +0000 UTC m=+643.122013623" Jul 10 18:36:39.482814 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:39.482775 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:39.542775 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:39.542739 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:39.710491 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:39.710456 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:41.505344 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:41.505300 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:42.864630 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:42.864588 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:43.615629 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.615593 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds"] Jul 10 18:36:43.615806 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.615766 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" containerName="manager" containerID="cri-o://7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600" gracePeriod=2 Jul 10 18:36:43.617898 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.617873 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:43.628730 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.628694 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds"] Jul 10 18:36:43.630651 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:36:43.630612 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wxnds\" is forbidden: User \"system:node:ip-10-0-138-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-224.ec2.internal' and this object" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:43.649256 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.649224 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw"] Jul 10 18:36:43.649573 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.649561 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" containerName="manager" Jul 10 18:36:43.649625 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.649574 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" containerName="manager" Jul 10 18:36:43.649670 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.649661 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" containerName="manager" Jul 10 18:36:43.652694 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.652670 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw" Jul 10 18:36:43.655348 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:36:43.655324 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wxnds\" is forbidden: User \"system:node:ip-10-0-138-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-224.ec2.internal' and this object" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:43.662329 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.662303 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw"] Jul 10 18:36:43.826910 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.826878 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vp76\" (UniqueName: \"kubernetes.io/projected/c5c7424b-6193-462d-a72d-779662589016-kube-api-access-8vp76\") pod \"limitador-operator-controller-manager-859d7b78bd-lqdcw\" (UID: \"c5c7424b-6193-462d-a72d-779662589016\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw" Jul 10 18:36:43.865484 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.865459 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:43.868337 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:36:43.868281 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wxnds\" is forbidden: User \"system:node:ip-10-0-138-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-224.ec2.internal' and this object" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:43.879218 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.879174 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cl7kc" Jul 10 18:36:43.922163 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:36:43.922118 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wxnds\" is forbidden: User \"system:node:ip-10-0-138-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-224.ec2.internal' and this object" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:43.926038 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.926009 2565 generic.go:356] "Generic (PLEG): container finished" podID="13d80b15-0300-4a9e-a59d-9b3808657862" containerID="7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600" exitCode=0 Jul 10 18:36:43.926154 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.926050 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:43.926154 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.926104 2565 scope.go:117] "RemoveContainer" containerID="7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600" Jul 10 18:36:43.927317 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.927301 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vp76\" (UniqueName: \"kubernetes.io/projected/c5c7424b-6193-462d-a72d-779662589016-kube-api-access-8vp76\") pod \"limitador-operator-controller-manager-859d7b78bd-lqdcw\" (UID: \"c5c7424b-6193-462d-a72d-779662589016\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw" Jul 10 18:36:43.929374 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:36:43.929353 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wxnds\" is forbidden: User \"system:node:ip-10-0-138-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-224.ec2.internal' and this object" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:43.935021 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.935000 2565 scope.go:117] "RemoveContainer" containerID="7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600" Jul 10 18:36:43.935443 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:36:43.935405 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600\": container with ID starting with 7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600 not found: ID does not exist" containerID="7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600" Jul 10 18:36:43.935553 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.935448 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600"} err="failed to get container status \"7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600\": rpc error: code = NotFound desc = could not find container \"7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600\": container with ID starting with 7d2a802869cc8bd33459e0b4b6e0bffd4155703d31a8f4a8403c0d12c8fa1600 not found: ID does not exist" Jul 10 18:36:43.936244 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.936179 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:43.938737 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.938713 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vp76\" (UniqueName: \"kubernetes.io/projected/c5c7424b-6193-462d-a72d-779662589016-kube-api-access-8vp76\") pod \"limitador-operator-controller-manager-859d7b78bd-lqdcw\" (UID: \"c5c7424b-6193-462d-a72d-779662589016\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw" Jul 10 18:36:43.998335 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:43.998300 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw" Jul 10 18:36:44.027741 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:44.027708 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9xkt\" (UniqueName: \"kubernetes.io/projected/13d80b15-0300-4a9e-a59d-9b3808657862-kube-api-access-l9xkt\") pod \"13d80b15-0300-4a9e-a59d-9b3808657862\" (UID: \"13d80b15-0300-4a9e-a59d-9b3808657862\") " Jul 10 18:36:44.029774 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:44.029742 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d80b15-0300-4a9e-a59d-9b3808657862-kube-api-access-l9xkt" (OuterVolumeSpecName: "kube-api-access-l9xkt") pod "13d80b15-0300-4a9e-a59d-9b3808657862" (UID: "13d80b15-0300-4a9e-a59d-9b3808657862"). InnerVolumeSpecName "kube-api-access-l9xkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:36:44.129007 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:44.128926 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9xkt\" (UniqueName: \"kubernetes.io/projected/13d80b15-0300-4a9e-a59d-9b3808657862-kube-api-access-l9xkt\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:36:44.135635 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:44.135613 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw"] Jul 10 18:36:44.137758 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:36:44.137734 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c7424b_6193_462d_a72d_779662589016.slice/crio-7c338b0906f98c9595fe82ddaacaca3fc5de790c2c6147ec744d1885e73e49d7 WatchSource:0}: Error finding container 7c338b0906f98c9595fe82ddaacaca3fc5de790c2c6147ec744d1885e73e49d7: Status 404 returned error can't find the container with id 7c338b0906f98c9595fe82ddaacaca3fc5de790c2c6147ec744d1885e73e49d7 Jul 10 18:36:44.237185 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:36:44.237146 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wxnds\" is forbidden: User \"system:node:ip-10-0-138-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-224.ec2.internal' and this object" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:44.932750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:44.932664 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw" event={"ID":"c5c7424b-6193-462d-a72d-779662589016","Type":"ContainerStarted","Data":"e9d86910e318b306579ae35dec7a1af2c154d48885595264145412f98761831c"} Jul 10 18:36:44.932750 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:44.932713 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw" event={"ID":"c5c7424b-6193-462d-a72d-779662589016","Type":"ContainerStarted","Data":"7c338b0906f98c9595fe82ddaacaca3fc5de790c2c6147ec744d1885e73e49d7"} Jul 10 18:36:44.933226 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:44.932763 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw" Jul 10 18:36:44.935392 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:36:44.935362 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wxnds\" is forbidden: User \"system:node:ip-10-0-138-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-224.ec2.internal' and this object" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:44.959299 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:44.959252 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw" podStartSLOduration=1.9592339 podStartE2EDuration="1.9592339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:36:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:36:44.956405362 +0000 UTC m=+650.162245414" watchObservedRunningTime="2026-07-10 18:36:44.9592339 +0000 UTC m=+650.165073955" Jul 10 18:36:45.108584 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:45.108544 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:45.205549 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:45.205456 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:45.294080 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:36:45.294040 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wxnds\" is forbidden: User \"system:node:ip-10-0-138-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-224.ec2.internal' and this object" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxnds" Jul 10 18:36:45.295209 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:45.295171 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13d80b15-0300-4a9e-a59d-9b3808657862" path="/var/lib/kubelet/pods/13d80b15-0300-4a9e-a59d-9b3808657862/volumes" Jul 10 18:36:45.485640 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:45.485548 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:45.586010 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:45.585971 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:49.080392 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:49.080349 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:49.178446 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:49.178407 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:52.326487 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:52.326446 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:52.871339 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:52.871304 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:52.971698 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:52.971658 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:55.941105 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:55.941074 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lqdcw" Jul 10 18:36:56.693578 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:56.693537 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:36:56.796893 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:36:56.796853 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:05.081205 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:05.081153 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:05.181574 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:05.181539 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:07.524738 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:07.524658 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:07.626770 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:07.626715 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:12.286714 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:12.286675 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:12.396810 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:12.396781 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:16.490327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:16.490276 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:16.591005 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:16.590967 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:24.234572 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:24.234543 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:24.421867 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:24.421789 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:25.299275 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:25.299244 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:25.399557 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:25.399525 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:25.989926 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:25.989890 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:26.092468 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.092418 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:26.190002 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.189963 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:26.339835 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.339757 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:37:26.343426 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.343407 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" Jul 10 18:37:26.356185 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.356164 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 18:37:26.356795 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.356778 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jl9r5\"" Jul 10 18:37:26.362529 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.362508 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:37:26.386602 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.386568 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:37:26.504592 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.504551 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b5f9a535-4ca2-49f5-8543-5c3d676021b4-config-file\") pod \"limitador-limitador-54d6d988d5-skvtr\" (UID: \"b5f9a535-4ca2-49f5-8543-5c3d676021b4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" Jul 10 18:37:26.504771 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.504622 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w87gp\" (UniqueName: \"kubernetes.io/projected/b5f9a535-4ca2-49f5-8543-5c3d676021b4-kube-api-access-w87gp\") pod \"limitador-limitador-54d6d988d5-skvtr\" (UID: \"b5f9a535-4ca2-49f5-8543-5c3d676021b4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" Jul 10 18:37:26.606220 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.606105 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b5f9a535-4ca2-49f5-8543-5c3d676021b4-config-file\") pod \"limitador-limitador-54d6d988d5-skvtr\" (UID: \"b5f9a535-4ca2-49f5-8543-5c3d676021b4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" Jul 10 18:37:26.606220 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.606161 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w87gp\" (UniqueName: \"kubernetes.io/projected/b5f9a535-4ca2-49f5-8543-5c3d676021b4-kube-api-access-w87gp\") pod \"limitador-limitador-54d6d988d5-skvtr\" (UID: \"b5f9a535-4ca2-49f5-8543-5c3d676021b4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" Jul 10 18:37:26.606786 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.606765 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b5f9a535-4ca2-49f5-8543-5c3d676021b4-config-file\") pod \"limitador-limitador-54d6d988d5-skvtr\" (UID: \"b5f9a535-4ca2-49f5-8543-5c3d676021b4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" Jul 10 18:37:26.614857 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.614822 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w87gp\" (UniqueName: \"kubernetes.io/projected/b5f9a535-4ca2-49f5-8543-5c3d676021b4-kube-api-access-w87gp\") pod \"limitador-limitador-54d6d988d5-skvtr\" (UID: \"b5f9a535-4ca2-49f5-8543-5c3d676021b4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" Jul 10 18:37:26.652903 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.652868 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" Jul 10 18:37:26.784042 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.784016 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:37:26.786373 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:37:26.786344 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f9a535_4ca2_49f5_8543_5c3d676021b4.slice/crio-f2fd38fed42c49b04392b83bdb559d6d0abb651139ba1961942d73f81dd43a21 WatchSource:0}: Error finding container f2fd38fed42c49b04392b83bdb559d6d0abb651139ba1961942d73f81dd43a21: Status 404 returned error can't find the container with id f2fd38fed42c49b04392b83bdb559d6d0abb651139ba1961942d73f81dd43a21 Jul 10 18:37:26.863018 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.862980 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:26.927180 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:26.927123 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:27.130682 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:27.130593 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" event={"ID":"b5f9a535-4ca2-49f5-8543-5c3d676021b4","Type":"ContainerStarted","Data":"f2fd38fed42c49b04392b83bdb559d6d0abb651139ba1961942d73f81dd43a21"} Jul 10 18:37:27.753421 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:27.753379 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:27.855123 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:27.855081 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:30.147416 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:30.147371 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" event={"ID":"b5f9a535-4ca2-49f5-8543-5c3d676021b4","Type":"ContainerStarted","Data":"3762a7193df7249216eb9259ba7b95a9b0fb46313b0a2ca04d8a5d6783150094"} Jul 10 18:37:30.147787 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:30.147455 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" Jul 10 18:37:30.164208 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:30.164137 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" podStartSLOduration=1.743241187 podStartE2EDuration="4.164120409s" totalImagesPullingTime="2.420879222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:37:30.162474317 +0000 UTC m=+695.368314382" watchObservedRunningTime="2026-07-10 18:37:30.164120409 +0000 UTC m=+695.369960463" Jul 10 18:37:32.494886 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:32.494848 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:32.598862 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:32.598834 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:41.153214 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:41.153174 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-skvtr" Jul 10 18:37:47.251573 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:47.251484 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:47.359365 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:47.359329 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:47.678112 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:37:47.678077 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:02.648004 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:02.647968 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:02.750350 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:02.750303 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:03.148875 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:03.148838 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7f8d8b4bdb-dng62"] Jul 10 18:38:03.155302 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:03.155275 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" Jul 10 18:38:03.157706 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:03.157677 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f8d8b4bdb-dng62"] Jul 10 18:38:03.158480 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:03.158459 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dshtt\"" Jul 10 18:38:03.228775 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:03.228734 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzqx7\" (UniqueName: \"kubernetes.io/projected/e832d471-157d-44e9-ae2d-fc44d0cfd185-kube-api-access-nzqx7\") pod \"authorino-7f8d8b4bdb-dng62\" (UID: \"e832d471-157d-44e9-ae2d-fc44d0cfd185\") " pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" Jul 10 18:38:03.329993 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:03.329949 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzqx7\" (UniqueName: \"kubernetes.io/projected/e832d471-157d-44e9-ae2d-fc44d0cfd185-kube-api-access-nzqx7\") pod \"authorino-7f8d8b4bdb-dng62\" (UID: \"e832d471-157d-44e9-ae2d-fc44d0cfd185\") " pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" Jul 10 18:38:03.339429 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:03.339393 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzqx7\" (UniqueName: \"kubernetes.io/projected/e832d471-157d-44e9-ae2d-fc44d0cfd185-kube-api-access-nzqx7\") pod \"authorino-7f8d8b4bdb-dng62\" (UID: \"e832d471-157d-44e9-ae2d-fc44d0cfd185\") " pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" Jul 10 18:38:03.352318 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:03.352288 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f8d8b4bdb-dng62"] Jul 10 18:38:03.352560 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:03.352549 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" Jul 10 18:38:03.490248 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:03.490222 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f8d8b4bdb-dng62"] Jul 10 18:38:03.492901 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:38:03.492871 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode832d471_157d_44e9_ae2d_fc44d0cfd185.slice/crio-77d437f240894271d0c2614955e0e444ababe9bbb99d2b87dbf645e2b4601812 WatchSource:0}: Error finding container 77d437f240894271d0c2614955e0e444ababe9bbb99d2b87dbf645e2b4601812: Status 404 returned error can't find the container with id 77d437f240894271d0c2614955e0e444ababe9bbb99d2b87dbf645e2b4601812 Jul 10 18:38:04.318352 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:04.318307 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" event={"ID":"e832d471-157d-44e9-ae2d-fc44d0cfd185","Type":"ContainerStarted","Data":"77d437f240894271d0c2614955e0e444ababe9bbb99d2b87dbf645e2b4601812"} Jul 10 18:38:07.335519 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:07.335484 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" event={"ID":"e832d471-157d-44e9-ae2d-fc44d0cfd185","Type":"ContainerStarted","Data":"b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79"} Jul 10 18:38:07.335997 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:07.335563 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" podUID="e832d471-157d-44e9-ae2d-fc44d0cfd185" containerName="authorino" containerID="cri-o://b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79" gracePeriod=30 Jul 10 18:38:07.351362 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:07.351316 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" podStartSLOduration=1.141939024 podStartE2EDuration="4.351301213s" totalImagesPullingTime="3.209362189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:38:07.34985416 +0000 UTC m=+732.555694216" watchObservedRunningTime="2026-07-10 18:38:07.351301213 +0000 UTC m=+732.557141267" Jul 10 18:38:07.588923 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:07.588865 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" Jul 10 18:38:07.671790 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:07.671745 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzqx7\" (UniqueName: \"kubernetes.io/projected/e832d471-157d-44e9-ae2d-fc44d0cfd185-kube-api-access-nzqx7\") pod \"e832d471-157d-44e9-ae2d-fc44d0cfd185\" (UID: \"e832d471-157d-44e9-ae2d-fc44d0cfd185\") " Jul 10 18:38:07.673971 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:07.673943 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e832d471-157d-44e9-ae2d-fc44d0cfd185-kube-api-access-nzqx7" (OuterVolumeSpecName: "kube-api-access-nzqx7") pod "e832d471-157d-44e9-ae2d-fc44d0cfd185" (UID: "e832d471-157d-44e9-ae2d-fc44d0cfd185"). InnerVolumeSpecName "kube-api-access-nzqx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:38:07.772509 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:07.772467 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzqx7\" (UniqueName: \"kubernetes.io/projected/e832d471-157d-44e9-ae2d-fc44d0cfd185-kube-api-access-nzqx7\") on node \"ip-10-0-138-224.ec2.internal\" DevicePath \"\"" Jul 10 18:38:08.341584 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:08.341543 2565 generic.go:356] "Generic (PLEG): container finished" podID="e832d471-157d-44e9-ae2d-fc44d0cfd185" containerID="b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79" exitCode=0 Jul 10 18:38:08.341584 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:08.341584 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" Jul 10 18:38:08.342073 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:08.341597 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" event={"ID":"e832d471-157d-44e9-ae2d-fc44d0cfd185","Type":"ContainerDied","Data":"b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79"} Jul 10 18:38:08.342073 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:08.341636 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f8d8b4bdb-dng62" event={"ID":"e832d471-157d-44e9-ae2d-fc44d0cfd185","Type":"ContainerDied","Data":"77d437f240894271d0c2614955e0e444ababe9bbb99d2b87dbf645e2b4601812"} Jul 10 18:38:08.342073 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:08.341658 2565 scope.go:117] "RemoveContainer" containerID="b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79" Jul 10 18:38:08.350757 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:08.350737 2565 scope.go:117] "RemoveContainer" containerID="b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79" Jul 10 18:38:08.351025 ip-10-0-138-224 kubenswrapper[2565]: E0710 18:38:08.351007 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79\": container with ID starting with b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79 not found: ID does not exist" containerID="b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79" Jul 10 18:38:08.351102 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:08.351040 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79"} err="failed to get container status \"b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79\": rpc error: code = NotFound desc = could not find container \"b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79\": container with ID starting with b1ff3179c7f296d1c1dfc07965c41dad206bd75a116669f619db16022f6e7c79 not found: ID does not exist" Jul 10 18:38:08.362242 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:08.362210 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f8d8b4bdb-dng62"] Jul 10 18:38:08.366031 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:08.366006 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7f8d8b4bdb-dng62"] Jul 10 18:38:09.294273 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:09.294240 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e832d471-157d-44e9-ae2d-fc44d0cfd185" path="/var/lib/kubelet/pods/e832d471-157d-44e9-ae2d-fc44d0cfd185/volumes" Jul 10 18:38:13.743785 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:13.743745 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:13.836017 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:13.835980 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:13.927548 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:13.927513 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:14.075771 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:14.075685 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:17.073217 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:17.073174 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:17.177865 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:17.177827 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:20.484856 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:20.484824 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:49.521613 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:49.521577 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:50.127799 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:50.127755 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:50.609828 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:50.609797 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:50.711794 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:50.711760 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:50.783226 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:50.783174 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:50.984751 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:38:50.984669 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:39:00.440158 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:00.440128 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:39:00.538610 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:00.538571 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:39:10.829146 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:10.829108 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:39:11.191381 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:11.191345 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:39:23.627237 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:23.627179 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:39:24.392919 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:24.392885 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:39:31.731592 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:31.731551 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:39:32.788658 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:32.788623 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:39:43.681707 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.681670 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8"] Jul 10 18:39:43.682335 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.682311 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e832d471-157d-44e9-ae2d-fc44d0cfd185" containerName="authorino" Jul 10 18:39:43.682335 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.682337 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e832d471-157d-44e9-ae2d-fc44d0cfd185" containerName="authorino" Jul 10 18:39:43.682484 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.682446 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e832d471-157d-44e9-ae2d-fc44d0cfd185" containerName="authorino" Jul 10 18:39:43.686090 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.686072 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.690597 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.690575 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 18:39:43.690718 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.690621 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 10 18:39:43.690718 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.690575 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 18:39:43.690832 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.690805 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fr6kk\"" Jul 10 18:39:43.697581 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.697559 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8"] Jul 10 18:39:43.725384 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.725351 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:39:43.850852 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.850810 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.851045 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.850885 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.851045 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.850916 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/031b8d3f-c3db-48c3-9ba7-560473916ff9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.851045 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.850934 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.851045 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.850975 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvqb5\" (UniqueName: \"kubernetes.io/projected/031b8d3f-c3db-48c3-9ba7-560473916ff9-kube-api-access-wvqb5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.851045 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.851016 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.952004 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.951920 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.952004 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.951976 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.952240 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.952015 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.952240 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.952041 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/031b8d3f-c3db-48c3-9ba7-560473916ff9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.952240 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.952066 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.952240 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.952094 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvqb5\" (UniqueName: \"kubernetes.io/projected/031b8d3f-c3db-48c3-9ba7-560473916ff9-kube-api-access-wvqb5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.952487 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.952464 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.952487 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.952479 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.952605 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.952537 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.954515 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.954488 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/031b8d3f-c3db-48c3-9ba7-560473916ff9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.954713 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.954696 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/031b8d3f-c3db-48c3-9ba7-560473916ff9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.960529 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.960503 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvqb5\" (UniqueName: \"kubernetes.io/projected/031b8d3f-c3db-48c3-9ba7-560473916ff9-kube-api-access-wvqb5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8\" (UID: \"031b8d3f-c3db-48c3-9ba7-560473916ff9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:43.999399 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:43.999363 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:44.139610 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:44.139574 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8"] Jul 10 18:39:44.142997 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:39:44.142969 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod031b8d3f_c3db_48c3_9ba7_560473916ff9.slice/crio-f3d491e5ce6f1442d045335369ea5535966f15dd51cdb937611bb7d9445c36b3 WatchSource:0}: Error finding container f3d491e5ce6f1442d045335369ea5535966f15dd51cdb937611bb7d9445c36b3: Status 404 returned error can't find the container with id f3d491e5ce6f1442d045335369ea5535966f15dd51cdb937611bb7d9445c36b3 Jul 10 18:39:44.803745 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:44.803706 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" event={"ID":"031b8d3f-c3db-48c3-9ba7-560473916ff9","Type":"ContainerStarted","Data":"f3d491e5ce6f1442d045335369ea5535966f15dd51cdb937611bb7d9445c36b3"} Jul 10 18:39:45.490324 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:45.490240 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:39:45.809860 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:45.809772 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" event={"ID":"031b8d3f-c3db-48c3-9ba7-560473916ff9","Type":"ContainerStarted","Data":"dda680d8a5f14ec12d04af277644ffcb886a44db3abdde1e521f8120b4cc1d41"} Jul 10 18:39:45.831321 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:45.831271 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" podStartSLOduration=1.834898107 podStartE2EDuration="2.831255236s" totalImagesPullingTime="996.357129ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:39:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:39:45.830125157 +0000 UTC m=+831.035965245" watchObservedRunningTime="2026-07-10 18:39:45.831255236 +0000 UTC m=+831.037095289" Jul 10 18:39:51.048668 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:51.048628 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:39:53.284021 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:53.283985 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:39:53.999901 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:53.999857 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:54.000375 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:53.999936 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:54.012310 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.012284 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:54.287761 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.287676 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k"] Jul 10 18:39:54.291063 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.291036 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.294342 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.294324 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 10 18:39:54.300897 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.300875 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k"] Jul 10 18:39:54.319152 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.319123 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:39:54.344355 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.344314 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.344560 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.344380 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.344560 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.344424 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbgc4\" (UniqueName: \"kubernetes.io/projected/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-kube-api-access-mbgc4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.344560 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.344489 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.344708 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.344567 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.344708 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.344598 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.445633 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.445598 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.445633 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.445639 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.445910 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.445771 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbgc4\" (UniqueName: \"kubernetes.io/projected/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-kube-api-access-mbgc4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.445910 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.445825 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.445910 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.445890 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.446076 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.445926 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.446169 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.446146 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.446359 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.446333 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.446480 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.446342 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.447983 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.447955 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.448317 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.448299 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.455489 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.455466 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbgc4\" (UniqueName: \"kubernetes.io/projected/9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d-kube-api-access-mbgc4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k\" (UID: \"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.600872 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.600769 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:39:54.737437 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.737399 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k"] Jul 10 18:39:54.739266 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:39:54.739241 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f3e0a72_0d1e_4a87_8ed9_146a3f5b972d.slice/crio-235353e5a8d91b45c6a254fb4998854d606bef7f5489df69c29668cf5b5ba9ef WatchSource:0}: Error finding container 235353e5a8d91b45c6a254fb4998854d606bef7f5489df69c29668cf5b5ba9ef: Status 404 returned error can't find the container with id 235353e5a8d91b45c6a254fb4998854d606bef7f5489df69c29668cf5b5ba9ef Jul 10 18:39:54.853775 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.853681 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" event={"ID":"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d","Type":"ContainerStarted","Data":"235353e5a8d91b45c6a254fb4998854d606bef7f5489df69c29668cf5b5ba9ef"} Jul 10 18:39:54.864953 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:54.864929 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8" Jul 10 18:39:55.381531 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:55.381494 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:39:55.434604 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:55.434566 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:39:55.866266 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:55.866221 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" event={"ID":"9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d","Type":"ContainerStarted","Data":"d43e9d850f2af311315d4d59f2bf0858841f9a2575f4bc918175c801af861d3c"} Jul 10 18:39:55.886552 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:39:55.886501 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" podStartSLOduration=1.704187708 podStartE2EDuration="1.886486382s" totalImagesPullingTime="182.298674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:39:55.88489957 +0000 UTC m=+841.090739625" watchObservedRunningTime="2026-07-10 18:39:55.886486382 +0000 UTC m=+841.092326436" Jul 10 18:40:01.592909 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:40:01.592870 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:40:04.601042 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:40:04.600999 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:40:04.601575 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:40:04.601059 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:40:04.613643 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:40:04.613615 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:40:04.919926 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:40:04.919898 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k" Jul 10 18:40:55.222238 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:40:55.222146 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:40:55.222778 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:40:55.222262 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:41:16.891995 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:16.891956 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67c5896d9f-8lbtb"] Jul 10 18:41:16.895333 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:16.895315 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67c5896d9f-8lbtb" Jul 10 18:41:16.899560 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:16.899539 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 10 18:41:16.899702 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:16.899686 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dshtt\"" Jul 10 18:41:16.904909 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:16.904885 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67c5896d9f-8lbtb"] Jul 10 18:41:16.997613 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:16.997572 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09e1916f-0ec5-4dd2-b547-de3da2ac9fac-tls-cert\") pod \"authorino-67c5896d9f-8lbtb\" (UID: \"09e1916f-0ec5-4dd2-b547-de3da2ac9fac\") " pod="kuadrant-system/authorino-67c5896d9f-8lbtb" Jul 10 18:41:16.997787 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:16.997644 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bqh7\" (UniqueName: \"kubernetes.io/projected/09e1916f-0ec5-4dd2-b547-de3da2ac9fac-kube-api-access-4bqh7\") pod \"authorino-67c5896d9f-8lbtb\" (UID: \"09e1916f-0ec5-4dd2-b547-de3da2ac9fac\") " pod="kuadrant-system/authorino-67c5896d9f-8lbtb" Jul 10 18:41:17.098607 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:17.098566 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09e1916f-0ec5-4dd2-b547-de3da2ac9fac-tls-cert\") pod \"authorino-67c5896d9f-8lbtb\" (UID: \"09e1916f-0ec5-4dd2-b547-de3da2ac9fac\") " pod="kuadrant-system/authorino-67c5896d9f-8lbtb" Jul 10 18:41:17.098811 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:17.098625 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqh7\" (UniqueName: \"kubernetes.io/projected/09e1916f-0ec5-4dd2-b547-de3da2ac9fac-kube-api-access-4bqh7\") pod \"authorino-67c5896d9f-8lbtb\" (UID: \"09e1916f-0ec5-4dd2-b547-de3da2ac9fac\") " pod="kuadrant-system/authorino-67c5896d9f-8lbtb" Jul 10 18:41:17.101376 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:17.101348 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09e1916f-0ec5-4dd2-b547-de3da2ac9fac-tls-cert\") pod \"authorino-67c5896d9f-8lbtb\" (UID: \"09e1916f-0ec5-4dd2-b547-de3da2ac9fac\") " pod="kuadrant-system/authorino-67c5896d9f-8lbtb" Jul 10 18:41:17.107602 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:17.107573 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bqh7\" (UniqueName: \"kubernetes.io/projected/09e1916f-0ec5-4dd2-b547-de3da2ac9fac-kube-api-access-4bqh7\") pod \"authorino-67c5896d9f-8lbtb\" (UID: \"09e1916f-0ec5-4dd2-b547-de3da2ac9fac\") " pod="kuadrant-system/authorino-67c5896d9f-8lbtb" Jul 10 18:41:17.206023 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:17.205950 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67c5896d9f-8lbtb" Jul 10 18:41:17.292464 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:17.292421 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:17.355051 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:17.354877 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67c5896d9f-8lbtb"] Jul 10 18:41:17.357891 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:41:17.357858 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09e1916f_0ec5_4dd2_b547_de3da2ac9fac.slice/crio-6769bf0ca0ae02fbc3648f901faf013949dcea9e759ae2a4dcab3556d2faff7c WatchSource:0}: Error finding container 6769bf0ca0ae02fbc3648f901faf013949dcea9e759ae2a4dcab3556d2faff7c: Status 404 returned error can't find the container with id 6769bf0ca0ae02fbc3648f901faf013949dcea9e759ae2a4dcab3556d2faff7c Jul 10 18:41:18.263978 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:18.263928 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67c5896d9f-8lbtb" event={"ID":"09e1916f-0ec5-4dd2-b547-de3da2ac9fac","Type":"ContainerStarted","Data":"e38922cc75a35858d08e6df635399c315cca371fc6547ee5cf31b9f67f283cdb"} Jul 10 18:41:18.264432 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:18.263990 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67c5896d9f-8lbtb" event={"ID":"09e1916f-0ec5-4dd2-b547-de3da2ac9fac","Type":"ContainerStarted","Data":"6769bf0ca0ae02fbc3648f901faf013949dcea9e759ae2a4dcab3556d2faff7c"} Jul 10 18:41:18.291734 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:18.291674 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-67c5896d9f-8lbtb" podStartSLOduration=1.92907951 podStartE2EDuration="2.291656591s" totalImagesPullingTime="362.577081ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:18.29047095 +0000 UTC m=+923.496311005" watchObservedRunningTime="2026-07-10 18:41:18.291656591 +0000 UTC m=+923.497496645" Jul 10 18:41:18.656020 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:18.655986 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:42.814452 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:42.814414 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:43.173405 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:43.173373 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:45.295030 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:45.294991 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:41:56.006726 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:41:56.006692 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:42:07.199882 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:42:07.199838 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:42:16.307651 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:42:16.307573 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:42:27.216094 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:42:27.216051 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:42:29.231122 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:42:29.231084 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:30.581307 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:42:30.581276 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:04.670140 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:04.670087 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:04.773430 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:04.773399 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:04.888012 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:04.887976 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:07.973532 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:07.973491 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:08.029212 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:08.029160 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:10.624614 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:10.624557 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:10.725615 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:10.725575 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:10.826043 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:10.826003 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:10.976387 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:10.976294 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:16.700956 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:16.700923 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:16.801479 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:16.801439 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:28.085890 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:28.085851 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:32.480742 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:32.480704 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:43:37.516822 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:37.516764 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:41.783814 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:41.783778 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:43:49.989182 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:49.989102 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:43:56.691264 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:43:56.691221 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:44:06.992989 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:44:06.992950 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:44:08.293233 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:44:08.293177 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:08.384251 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:44:08.384217 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:16.396148 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:44:16.396108 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:44:27.993043 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:44:27.993007 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:44:37.991991 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:44:37.991953 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:45:00.092146 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:45:00.092105 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:45:11.889715 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:45:11.889678 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:45:34.093371 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:45:34.093337 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:45:44.895489 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:45:44.895443 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:45:55.252089 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:45:55.252056 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:45:55.253578 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:45:55.253553 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:46:01.595848 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:46:01.595810 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:46:07.829170 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:46:07.829137 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:11.486290 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:46:11.486245 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:46:20.890013 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:46:20.889967 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:46:30.691462 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:46:30.691424 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:46:40.187649 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:46:40.187612 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:48:26.598007 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:48:26.597925 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:48:37.385146 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:48:37.385098 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:48:47.587853 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:48:47.587809 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:48:52.542340 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:48:52.542289 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:56.596575 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:48:56.596537 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:49:06.990852 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:49:06.990815 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:49:15.296064 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:49:15.296033 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:49:24.781116 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:49:24.781078 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:49:35.188425 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:49:35.188384 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:49:43.790230 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:49:43.790182 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:49:55.083971 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:49:55.083882 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:50:06.595056 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:50:06.595005 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:50:16.191806 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:50:16.191756 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:50:32.290759 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:50:32.290714 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:50:41.188647 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:50:41.188603 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:50:52.891958 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:50:52.891913 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:50:55.282508 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:50:55.282481 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:50:55.285112 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:50:55.285080 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:51:05.094638 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:51:05.094598 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:51:20.486749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:51:20.486710 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:51:30.287224 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:51:30.287176 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:51:39.488011 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:51:39.487975 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:51:49.279002 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:51:49.278965 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:51:50.090781 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:51:50.090740 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:51:58.991513 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:51:58.991476 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:52:08.792319 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:52:08.792278 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:52:21.505211 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:52:21.502478 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:52:30.398376 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:52:30.398336 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:52:35.023813 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:52:35.023767 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:39.992798 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:52:39.992764 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:52:43.212735 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:52:43.212696 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:43.319749 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:52:43.319709 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:52.515523 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:52:52.513262 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:53:01.815135 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:53:01.815093 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:53:08.803237 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:53:08.803188 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:53:25.022057 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:53:25.022020 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:53:37.613071 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:53:37.613029 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:53:50.686397 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:53:50.686360 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:53:53.132661 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:53:53.132620 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:58.512995 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:53:58.512956 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:54:04.450229 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:54:04.450182 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:54:06.725847 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:54:06.725804 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:54:11.851669 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:54:11.851636 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:54:15.802770 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:54:15.802737 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-skvtr"] Jul 10 18:55:55.315046 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:55:55.315002 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:55:55.319544 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:55:55.319521 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ft2m_3a8ca608-59d8-4480-925d-00d01e6eda10/ovn-acl-logging/0.log" Jul 10 18:57:17.135801 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:17.135756 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67c5896d9f-8lbtb_09e1916f-0ec5-4dd2-b547-de3da2ac9fac/authorino/0.log" Jul 10 18:57:22.667243 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:22.667212 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5488d6dcc-wvmq8_e6e67385-387b-4b0f-944b-d8efa2f2c13c/copy-manifests/0.log" Jul 10 18:57:22.681209 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:22.681120 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5488d6dcc-wvmq8_e6e67385-387b-4b0f-944b-d8efa2f2c13c/manager/0.log" Jul 10 18:57:24.518020 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:24.517980 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67c5896d9f-8lbtb_09e1916f-0ec5-4dd2-b547-de3da2ac9fac/authorino/0.log" Jul 10 18:57:24.745981 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:24.745949 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cl7kc_31e94cf1-2771-4f25-b24b-a54fa25b1c1f/manager/0.log" Jul 10 18:57:25.266120 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:25.266089 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-skvtr_b5f9a535-4ca2-49f5-8543-5c3d676021b4/limitador/0.log" Jul 10 18:57:25.382939 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:25.382905 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lqdcw_c5c7424b-6193-462d-a72d-779662589016/manager/0.log" Jul 10 18:57:25.973702 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:25.973674 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd47df98c-9fxzx_c26b438b-cb32-4cd7-b3d0-c1b11b483bd5/kube-auth-proxy/0.log" Jul 10 18:57:27.234699 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:27.234670 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt6tv8_031b8d3f-c3db-48c3-9ba7-560473916ff9/main/0.log" Jul 10 18:57:27.459747 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:27.459716 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-sdg8k_9f3e0a72-0d1e-4a87-8ed9-146a3f5b972d/main/0.log" Jul 10 18:57:31.727905 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:31.727865 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:57:31.800611 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:31.800576 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" limit=1024 Jul 10 18:57:34.451725 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:34.451697 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8kx2z_098c1a69-137c-45c4-a356-ed61de961a17/global-pull-secret-syncer/0.log" Jul 10 18:57:34.542442 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:34.542416 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-f6s8j_8feead6f-4012-48ba-88db-2a3e0937d86b/konnectivity-agent/0.log" Jul 10 18:57:34.674585 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:34.674555 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-224.ec2.internal_9979441dfb575ea6a2636357cc791013/haproxy/0.log" Jul 10 18:57:40.042038 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:40.042000 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67c5896d9f-8lbtb_09e1916f-0ec5-4dd2-b547-de3da2ac9fac/authorino/0.log" Jul 10 18:57:40.113561 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:40.113532 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cl7kc_31e94cf1-2771-4f25-b24b-a54fa25b1c1f/manager/0.log" Jul 10 18:57:40.315936 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:40.315851 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-skvtr_b5f9a535-4ca2-49f5-8543-5c3d676021b4/limitador/0.log" Jul 10 18:57:40.366079 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:40.366047 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lqdcw_c5c7424b-6193-462d-a72d-779662589016/manager/0.log" Jul 10 18:57:42.050063 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:42.050038 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-dw5tc_a83a3d6f-a993-4fc5-ac83-a8b2ab4b028a/cluster-monitoring-operator/0.log" Jul 10 18:57:42.202611 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:42.202584 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-c89ffd77f-v4q2w_a3198dde-3bee-4659-b217-35286c1c7611/metrics-server/0.log" Jul 10 18:57:42.385528 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:42.385493 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ltvfr_4d9c257e-3e4b-4017-a462-6c37c213eddd/node-exporter/0.log" Jul 10 18:57:42.422641 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:42.422610 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ltvfr_4d9c257e-3e4b-4017-a462-6c37c213eddd/kube-rbac-proxy/0.log" Jul 10 18:57:42.450516 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:42.450493 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ltvfr_4d9c257e-3e4b-4017-a462-6c37c213eddd/init-textfile/0.log" Jul 10 18:57:42.772690 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:42.772610 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6b11dab4-de81-4582-8c66-6584ce5e5417/prometheus/0.log" Jul 10 18:57:42.808912 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:42.808884 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6b11dab4-de81-4582-8c66-6584ce5e5417/config-reloader/0.log" Jul 10 18:57:42.854055 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:42.854025 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6b11dab4-de81-4582-8c66-6584ce5e5417/thanos-sidecar/0.log" Jul 10 18:57:42.896134 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:42.896107 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6b11dab4-de81-4582-8c66-6584ce5e5417/kube-rbac-proxy-web/0.log" Jul 10 18:57:42.939970 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:42.939928 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6b11dab4-de81-4582-8c66-6584ce5e5417/kube-rbac-proxy/0.log" Jul 10 18:57:42.982862 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:42.982833 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6b11dab4-de81-4582-8c66-6584ce5e5417/kube-rbac-proxy-thanos/0.log" Jul 10 18:57:43.026953 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:43.026881 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6b11dab4-de81-4582-8c66-6584ce5e5417/init-config-reloader/0.log" Jul 10 18:57:43.078429 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:43.078398 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rz96w_603dd2d0-e930-4d54-83ad-090a381b2846/prometheus-operator/0.log" Jul 10 18:57:43.122765 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:43.122731 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rz96w_603dd2d0-e930-4d54-83ad-090a381b2846/kube-rbac-proxy/0.log" Jul 10 18:57:43.177761 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:43.177732 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-vg962_7147b18c-c060-410f-b82f-27beaeea9f60/prometheus-operator-admission-webhook/0.log" Jul 10 18:57:43.320341 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:43.320256 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bf7c7f8f7-rlpl6_9493119a-77ed-47bf-b4bc-2e1315e24e9b/thanos-query/0.log" Jul 10 18:57:43.368946 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:43.368918 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bf7c7f8f7-rlpl6_9493119a-77ed-47bf-b4bc-2e1315e24e9b/kube-rbac-proxy-web/0.log" Jul 10 18:57:43.413840 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:43.413815 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bf7c7f8f7-rlpl6_9493119a-77ed-47bf-b4bc-2e1315e24e9b/kube-rbac-proxy/0.log" Jul 10 18:57:43.462751 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:43.462721 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bf7c7f8f7-rlpl6_9493119a-77ed-47bf-b4bc-2e1315e24e9b/prom-label-proxy/0.log" Jul 10 18:57:43.506407 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:43.506378 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bf7c7f8f7-rlpl6_9493119a-77ed-47bf-b4bc-2e1315e24e9b/kube-rbac-proxy-rules/0.log" Jul 10 18:57:43.538977 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:43.538947 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bf7c7f8f7-rlpl6_9493119a-77ed-47bf-b4bc-2e1315e24e9b/kube-rbac-proxy-metrics/0.log" Jul 10 18:57:44.524323 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.524283 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8"] Jul 10 18:57:44.528034 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.528009 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.530825 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.530781 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zvmwd\"/\"openshift-service-ca.crt\"" Jul 10 18:57:44.530825 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.530821 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zvmwd\"/\"default-dockercfg-b4dbf\"" Jul 10 18:57:44.531071 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.530824 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zvmwd\"/\"kube-root-ca.crt\"" Jul 10 18:57:44.537327 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.537304 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8"] Jul 10 18:57:44.616688 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.616651 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d82kc\" (UniqueName: \"kubernetes.io/projected/fca721de-b1e9-4dae-92d9-0343801ed727-kube-api-access-d82kc\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.616688 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.616688 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-sys\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.616901 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.616766 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-proc\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.616901 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.616783 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-lib-modules\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.616901 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.616808 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-podres\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.718116 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.718073 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-proc\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.718116 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.718115 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-lib-modules\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.718367 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.718151 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-podres\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.718367 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.718228 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-proc\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.718367 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.718237 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d82kc\" (UniqueName: \"kubernetes.io/projected/fca721de-b1e9-4dae-92d9-0343801ed727-kube-api-access-d82kc\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.718367 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.718275 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-sys\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.718367 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.718326 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-lib-modules\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.718539 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.718372 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-sys\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.718539 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.718374 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fca721de-b1e9-4dae-92d9-0343801ed727-podres\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.726160 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.726128 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d82kc\" (UniqueName: \"kubernetes.io/projected/fca721de-b1e9-4dae-92d9-0343801ed727-kube-api-access-d82kc\") pod \"perf-node-gather-daemonset-nlkj8\" (UID: \"fca721de-b1e9-4dae-92d9-0343801ed727\") " pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.730758 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.730740 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-vt5c2_5bc9f8f6-2ed4-4cce-ac5c-2f4f1ce093ac/networking-console-plugin/0.log" Jul 10 18:57:44.840268 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.840152 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:44.981953 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:44.981923 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8"] Jul 10 18:57:44.983582 ip-10-0-138-224 kubenswrapper[2565]: W0710 18:57:44.983552 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfca721de_b1e9_4dae_92d9_0343801ed727.slice/crio-14f3df89cf91ea2b2b356a218e99f0c77d33c05a5ea462c3d057b6691d6825b0 WatchSource:0}: Error finding container 14f3df89cf91ea2b2b356a218e99f0c77d33c05a5ea462c3d057b6691d6825b0: Status 404 returned error can't find the container with id 14f3df89cf91ea2b2b356a218e99f0c77d33c05a5ea462c3d057b6691d6825b0 Jul 10 18:57:45.136477 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:45.136428 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" event={"ID":"fca721de-b1e9-4dae-92d9-0343801ed727","Type":"ContainerStarted","Data":"e9905827dccd0c0f281daba4373870e52705c3ea5e5ef921f226cbf203a6c9e4"} Jul 10 18:57:45.136664 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:45.136494 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" event={"ID":"fca721de-b1e9-4dae-92d9-0343801ed727","Type":"ContainerStarted","Data":"14f3df89cf91ea2b2b356a218e99f0c77d33c05a5ea462c3d057b6691d6825b0"} Jul 10 18:57:45.136664 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:45.136609 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:45.154632 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:45.154581 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" podStartSLOduration=1.154562718 podStartE2EDuration="1.154562718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:57:45.153651939 +0000 UTC m=+1910.359491996" watchObservedRunningTime="2026-07-10 18:57:45.154562718 +0000 UTC m=+1910.360402807" Jul 10 18:57:45.887714 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:45.887682 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-hsvxt_9c12efa8-f9e8-46a2-8ad3-3fa3e645dcde/download-server/0.log" Jul 10 18:57:46.483880 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:46.483851 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-kq2db_939c9e48-d662-4d9d-96c8-b14367001619/volume-data-source-validator/0.log" Jul 10 18:57:47.544173 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:47.544146 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5ft29_06300a4b-433b-4fa0-b969-78d216179a3b/dns/0.log" Jul 10 18:57:47.570948 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:47.570915 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5ft29_06300a4b-433b-4fa0-b969-78d216179a3b/kube-rbac-proxy/0.log" Jul 10 18:57:47.702156 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:47.702131 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7bh5c_f0d9f5e5-cd7b-4641-b584-1bb2dc6c9694/dns-node-resolver/0.log" Jul 10 18:57:48.388779 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:48.388748 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qjh75_001a040d-8e63-4eac-8af4-5ce45a0e4914/node-ca/0.log" Jul 10 18:57:49.498140 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:49.498109 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd47df98c-9fxzx_c26b438b-cb32-4cd7-b3d0-c1b11b483bd5/kube-auth-proxy/0.log" Jul 10 18:57:50.426268 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:50.426238 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-psq6h_c5ff392a-52f5-4430-9b45-c07ecc3445ec/serve-healthcheck-canary/0.log" Jul 10 18:57:50.962815 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:50.962784 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-57kb4_aacd5c76-7b3e-4ad4-817c-279d66dae83e/insights-operator/0.log" Jul 10 18:57:50.963271 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:50.963123 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-57kb4_aacd5c76-7b3e-4ad4-817c-279d66dae83e/insights-operator/1.log" Jul 10 18:57:50.985235 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:50.985186 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9ksnh_9d338c47-f51e-4517-98f1-9f4763a838a9/kube-rbac-proxy/0.log" Jul 10 18:57:51.013763 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:51.013733 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9ksnh_9d338c47-f51e-4517-98f1-9f4763a838a9/exporter/0.log" Jul 10 18:57:51.043405 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:51.043380 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9ksnh_9d338c47-f51e-4517-98f1-9f4763a838a9/extractor/0.log" Jul 10 18:57:51.151398 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:51.151371 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zvmwd/perf-node-gather-daemonset-nlkj8" Jul 10 18:57:53.718250 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:53.718214 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5488d6dcc-wvmq8_e6e67385-387b-4b0f-944b-d8efa2f2c13c/manager/0.log" Jul 10 18:57:53.739304 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:53.739276 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5488d6dcc-wvmq8_e6e67385-387b-4b0f-944b-d8efa2f2c13c/copy-manifests/0.log" Jul 10 18:57:54.703162 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:54.703072 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd47df98c-9fxzx_c26b438b-cb32-4cd7-b3d0-c1b11b483bd5/kube-auth-proxy/0.log" Jul 10 18:57:55.159922 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:55.159890 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-67d57b55c8-bnm5d_5771e988-ea9a-4da1-89fa-4bbf0bb5ac7f/manager/0.log" Jul 10 18:57:56.216057 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:57:56.216022 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd47df98c-9fxzx_c26b438b-cb32-4cd7-b3d0-c1b11b483bd5/kube-auth-proxy/0.log" Jul 10 18:58:00.793209 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:00.793158 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-6kgms_9b56e500-4923-4ab3-8b46-052e2a42839b/kube-storage-version-migrator-operator/1.log" Jul 10 18:58:00.794180 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:00.794158 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-6kgms_9b56e500-4923-4ab3-8b46-052e2a42839b/kube-storage-version-migrator-operator/0.log" Jul 10 18:58:02.070702 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:02.070671 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdnl4_ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5/kube-multus-additional-cni-plugins/0.log" Jul 10 18:58:02.097055 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:02.097023 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdnl4_ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5/egress-router-binary-copy/0.log" Jul 10 18:58:02.124163 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:02.124133 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdnl4_ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5/cni-plugins/0.log" Jul 10 18:58:02.147736 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:02.147709 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdnl4_ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5/bond-cni-plugin/0.log" Jul 10 18:58:02.173134 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:02.173044 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdnl4_ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5/routeoverride-cni/0.log" Jul 10 18:58:02.197454 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:02.197423 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdnl4_ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5/whereabouts-cni-bincopy/0.log" Jul 10 18:58:02.224347 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:02.224312 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vdnl4_ddf79a51-c7d4-47a8-8a87-8d6ccd4e90d5/whereabouts-cni/0.log" Jul 10 18:58:02.447140 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:02.447051 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q8vll_15c9edb2-53dc-4360-a559-1e478b3f98f2/kube-multus/0.log" Jul 10 18:58:02.654810 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:02.654777 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tvbm5_24908795-9d74-4fec-bb0b-859f3b202d44/network-metrics-daemon/0.log" Jul 10 18:58:02.679343 ip-10-0-138-224 kubenswrapper[2565]: I0710 18:58:02.679299 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tvbm5_24908795-9d74-4fec-bb0b-859f3b202d44/kube-rbac-proxy/0.log"