Jul 02 13:43:53.559318 ip-10-0-137-40 systemd[1]: Starting Kubernetes Kubelet... Jul 02 13:43:53.938029 ip-10-0-137-40 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 13:43:53.938029 ip-10-0-137-40 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 13:43:53.938029 ip-10-0-137-40 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 13:43:53.938029 ip-10-0-137-40 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 13:43:53.938029 ip-10-0-137-40 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 13:43:53.938681 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.938105 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 13:43:53.942425 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942403 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:43:53.942425 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942419 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:43:53.942425 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942424 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:43:53.942425 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942427 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:43:53.942425 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942430 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:43:53.942425 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942432 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:43:53.942425 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942435 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942439 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942442 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942445 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942448 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942450 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942453 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942456 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942458 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942461 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942463 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942466 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942468 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942471 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942479 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942482 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942484 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942487 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942490 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942492 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:43:53.942671 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942494 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942497 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942500 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942502 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942505 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942507 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942510 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942515 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942519 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942523 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942526 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942529 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942532 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942536 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942539 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942542 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942546 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942549 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942551 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:43:53.943160 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942554 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942557 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942559 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942562 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942564 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942567 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942570 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942572 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942575 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942578 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942580 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942583 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942585 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942588 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942590 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942593 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942597 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942600 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942602 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942605 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:43:53.943619 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942607 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942610 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942612 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942615 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942618 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942620 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942622 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942626 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942629 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942631 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942635 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942639 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942643 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942646 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942650 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942653 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942656 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942659 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942661 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:43:53.944099 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942664 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942667 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942670 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942672 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942675 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942678 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942681 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942684 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942687 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942689 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942692 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942694 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942697 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942699 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.942702 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943401 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943407 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943410 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943413 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943415 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:43:53.944579 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943419 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943422 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943424 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943427 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943430 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943433 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943436 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943438 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943441 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943443 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943446 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943449 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943451 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943455 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943459 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943462 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943464 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943467 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943470 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943472 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:43:53.945061 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943475 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943477 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943480 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943483 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943485 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943488 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943490 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943493 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943495 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943498 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943501 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943504 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943507 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943510 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943513 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943516 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943519 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943521 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943525 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:43:53.945578 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943530 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943533 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943536 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943539 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943541 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943544 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943547 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943549 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943552 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943554 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943557 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943559 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943562 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943564 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943567 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943570 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943573 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943576 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943578 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:43:53.946087 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943581 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943599 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943602 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943605 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943608 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943611 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943614 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943617 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943620 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943622 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943625 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943628 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943631 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943635 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943638 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943640 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943643 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943645 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943649 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:43:53.946557 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943653 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943655 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943658 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943661 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943663 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943666 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943668 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943670 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943673 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943676 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943678 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943681 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943683 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943686 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943688 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943690 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.943693 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944063 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944068 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944071 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944074 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:43:53.947015 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944077 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944080 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944082 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944085 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944088 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944091 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944093 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944095 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944098 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944101 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944103 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944106 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944109 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944111 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944114 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944116 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944119 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944122 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944124 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944127 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:43:53.947527 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944129 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944132 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944135 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944137 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944140 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944143 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944145 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944147 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944150 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944153 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944155 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944158 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944161 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944163 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944166 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944168 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944170 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944173 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944176 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944178 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:43:53.948095 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944181 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944183 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944188 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944191 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944194 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944197 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944200 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944203 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944206 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944209 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944212 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944215 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944217 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944220 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944223 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944225 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944228 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944231 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944233 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:43:53.948580 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944236 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944239 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944241 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944244 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944247 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944250 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944255 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944257 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944260 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944263 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944265 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944268 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944271 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944274 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944276 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944279 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944281 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944284 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944286 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:43:53.949089 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944289 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944291 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944295 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944297 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944300 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944302 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944305 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944307 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944310 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944314 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944317 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944320 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944322 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944325 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944328 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944333 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.944335 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944408 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944415 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944421 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 13:43:53.949565 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944425 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944429 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944432 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944436 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944440 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944444 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944447 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944450 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944454 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944460 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944463 2561 flags.go:64] FLAG: --cgroup-root="" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944466 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944469 2561 flags.go:64] FLAG: --client-ca-file="" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944471 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944474 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944479 2561 flags.go:64] FLAG: --cluster-domain="" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944482 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944485 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944488 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944492 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944496 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944498 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944502 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 13:43:53.950103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944505 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944508 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944511 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944514 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944519 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944522 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944526 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944529 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944532 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944534 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944538 2561 flags.go:64] FLAG: --enable-server="true" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944541 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944545 2561 flags.go:64] FLAG: --event-burst="100" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944548 2561 flags.go:64] FLAG: --event-qps="50" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944551 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944554 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944557 2561 flags.go:64] FLAG: --eviction-hard="" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944561 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944565 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944568 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944571 2561 flags.go:64] FLAG: --eviction-soft="" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944574 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944577 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944580 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944583 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 13:43:53.950653 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944586 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944589 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944592 2561 flags.go:64] FLAG: --feature-gates="" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944596 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944599 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944602 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944605 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944608 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944611 2561 flags.go:64] FLAG: --help="false" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944614 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-137-40.ec2.internal" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944617 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944621 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944624 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944627 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944630 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944633 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944636 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944638 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944641 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944644 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944647 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944650 2561 flags.go:64] FLAG: --kube-reserved="" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944652 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944656 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 13:43:53.951305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944659 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944664 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944667 2561 flags.go:64] FLAG: --lock-file="" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944670 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944673 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944676 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944681 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944684 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944686 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944689 2561 flags.go:64] FLAG: --logging-format="text" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944692 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944695 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944698 2561 flags.go:64] FLAG: --manifest-url="" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944701 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944705 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944708 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944712 2561 flags.go:64] FLAG: --max-pods="110" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944715 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944718 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944722 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944725 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944728 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944730 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944733 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944740 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 13:43:53.951891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944743 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944746 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944749 2561 flags.go:64] FLAG: --pod-cidr="" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944752 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944756 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944759 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944762 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944766 2561 flags.go:64] FLAG: --port="10250" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944769 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944773 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e390b16236eea6a8" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944776 2561 flags.go:64] FLAG: --qos-reserved="" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944779 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944782 2561 flags.go:64] FLAG: --register-node="true" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944784 2561 flags.go:64] FLAG: --register-with-taints="" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944788 2561 flags.go:64] FLAG: --registry-burst="10" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944791 2561 flags.go:64] FLAG: --registry-qps="5" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944794 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944796 2561 flags.go:64] FLAG: --reserved-memory="" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944800 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944803 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944806 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944808 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944811 2561 flags.go:64] FLAG: --runonce="false" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944814 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944817 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 13:43:53.952504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944820 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944824 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944827 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944830 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944832 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944835 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944838 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944841 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944844 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944846 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944849 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944852 2561 flags.go:64] FLAG: --system-cgroups="" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944855 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944860 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944863 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944866 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944872 2561 flags.go:64] FLAG: --tls-min-version="" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944874 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944877 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944880 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944883 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944886 2561 flags.go:64] FLAG: --v="2" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944889 2561 flags.go:64] FLAG: --version="false" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944893 2561 flags.go:64] FLAG: --vmodule="" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944897 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 13:43:53.953131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.944900 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945020 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945024 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945028 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945032 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945048 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945052 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945054 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945060 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945063 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945067 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945070 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945073 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945076 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945078 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945081 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945084 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945086 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945089 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:43:53.953801 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945091 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945094 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945096 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945099 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945103 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945106 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945108 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945111 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945114 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945116 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945119 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945121 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945124 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945127 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945129 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945132 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945134 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945136 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945139 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:43:53.954279 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945141 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945144 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945148 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945150 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945153 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945155 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945158 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945160 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945163 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945165 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945168 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945170 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945173 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945175 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945178 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945180 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945183 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945187 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945190 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945193 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:43:53.954737 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945195 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945198 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945200 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945203 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945205 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945207 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945210 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945213 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945216 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945219 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945221 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945224 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945226 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945229 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945233 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945235 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945238 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945240 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945243 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945246 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:43:53.955253 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945248 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945251 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945253 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945256 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945258 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945261 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945263 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945265 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945268 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945272 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945274 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945277 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945281 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945284 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945288 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945291 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945293 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945297 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945299 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945302 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:43:53.955743 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945305 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.945308 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.945924 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.952881 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.952895 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.953673 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.953685 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953766 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953773 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953777 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953780 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953785 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953789 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953792 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:43:53.956238 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953795 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953798 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953801 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953803 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953806 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953809 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953812 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953815 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953818 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953827 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953830 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953832 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953835 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953837 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953840 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953842 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953845 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953848 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953850 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953853 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:43:53.956583 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953856 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953858 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953862 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953865 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953868 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953871 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953873 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953876 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953878 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953881 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953884 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953886 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953889 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953891 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953894 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953897 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953900 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953902 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953905 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:43:53.957281 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953907 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953910 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953912 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953921 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953924 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953926 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953928 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953931 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953934 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953936 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953942 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953945 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953947 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953950 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953952 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953955 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953957 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953960 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953964 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:43:53.957741 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953968 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953970 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953973 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953975 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953978 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953980 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953983 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953986 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953988 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953991 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953993 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953996 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.953998 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954001 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954003 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954006 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954008 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954016 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954019 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:43:53.958244 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954021 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954024 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954026 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954029 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954031 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954034 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954056 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954059 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954061 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954064 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954066 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954069 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954071 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954074 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954076 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:43:53.958694 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.954081 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954231 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954235 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954238 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954240 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954243 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954246 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954249 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954251 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954254 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954257 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954259 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954262 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954264 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954267 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954269 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954279 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954282 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954285 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954287 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:43:53.959076 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954290 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954292 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954295 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954298 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954300 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954303 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954306 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954309 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954313 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954315 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954318 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954320 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954323 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954325 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954328 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954330 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954333 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954335 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954338 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:43:53.959548 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954340 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954343 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954345 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954348 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954350 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954353 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954355 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954357 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954360 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954363 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954374 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954377 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954380 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954382 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954385 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954388 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954390 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954393 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954395 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:43:53.960012 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954398 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954402 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954405 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954407 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954410 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954412 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954415 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954418 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954420 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954423 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954425 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954428 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954430 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954433 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954435 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954438 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954440 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954443 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954445 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954448 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:43:53.960493 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954451 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954453 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954456 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954458 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954466 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954469 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954471 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954473 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954476 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954478 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954481 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954483 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954488 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954491 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954494 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954497 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954501 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954503 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954506 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954509 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:43:53.960972 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954512 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:43:53.961500 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:53.954514 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:43:53.961500 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.954519 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 13:43:53.961500 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.954635 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 13:43:53.961500 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.956813 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 13:43:53.961500 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.958090 2561 server.go:1019] "Starting client certificate rotation" Jul 02 13:43:53.961500 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.958199 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 13:43:53.961500 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.958242 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 13:43:53.961500 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.960675 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 13:43:53.962219 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.962203 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 13:43:53.972683 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.972663 2561 log.go:25] "Validated CRI v1 runtime API" Jul 02 13:43:53.974980 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.974961 2561 log.go:25] "Validated CRI v1 image API" Jul 02 13:43:53.975751 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.975729 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 13:43:53.978067 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.978023 2561 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 976483ae-707f-4a01-8b93-69bf36d88f94:/dev/nvme0n1p4 af4e2cba-740c-4ad3-b96f-b8821fd8346f:/dev/nvme0n1p3] Jul 02 13:43:53.978146 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.978067 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 13:43:53.984005 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.983897 2561 manager.go:216] Machine: {Timestamp:2026-07-02 13:43:53.982672228 +0000 UTC m=+0.309233274 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098941 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27909e1cc8157fda8e5cd835a5c638 SystemUUID:ec27909e-1cc8-157f-da8e-5cd835a5c638 BootID:5841e8b9-d184-4fe6-bc24-d5318bcf4df9 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:35:b7:78:69:1b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:35:b7:78:69:1b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:34:cd:7e:d9:dd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 13:43:53.984005 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.983997 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 13:43:53.984161 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.984123 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 13:43:53.987084 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.987053 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 13:43:53.987221 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.987087 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-40.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 13:43:53.988654 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.988641 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 13:43:53.988700 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.988658 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 13:43:53.988700 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.988665 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 13:43:53.989259 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.989249 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 13:43:53.990047 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.990027 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 13:43:53.990923 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.990907 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 13:43:53.992961 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.992950 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 02 13:43:53.993079 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.993069 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 13:43:53.993123 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.993100 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 02 13:43:53.993123 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.993114 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 13:43:53.993206 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.993127 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 13:43:53.993206 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.993137 2561 kubelet.go:398] "Adding apiserver pod source" Jul 02 13:43:53.993206 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.993146 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 13:43:53.993579 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.993567 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 13:43:53.993639 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.993587 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 13:43:53.995841 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.995812 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 13:43:53.997336 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.997319 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 13:43:53.997380 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.997353 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 13:43:53.997985 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.997973 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 13:43:53.998075 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.997989 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 13:43:53.998075 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.997995 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 13:43:53.998075 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.998001 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 13:43:53.998075 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.998009 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 13:43:53.998075 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.998015 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 13:43:53.998261 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.998195 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 13:43:53.998261 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.998201 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 13:43:53.998261 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.998209 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 13:43:53.998261 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.998214 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 13:43:53.998261 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.998223 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 13:43:53.998261 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.998231 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 13:43:53.999063 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.999050 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 13:43:53.999113 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:53.999065 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 13:43:54.002291 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.002275 2561 server.go:1268] "Started kubelet" Jul 02 13:43:54.002392 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.002362 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 13:43:54.002454 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.002419 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 13:43:54.002542 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.002520 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 13:43:54.003060 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.003023 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-40.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 13:43:54.003117 ip-10-0-137-40 systemd[1]: Started Kubernetes Kubelet. Jul 02 13:43:54.006612 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.006577 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 13:43:54.006802 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.006767 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 13:43:54.006996 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.006799 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 13:43:54.008234 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.008214 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 13:43:54.008863 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.008847 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 02 13:43:54.010880 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.010029 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-40.ec2.internal.18be7ce6fc46577d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-40.ec2.internal,UID:ip-10-0-137-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-40.ec2.internal,},FirstTimestamp:2026-07-02 13:43:54.002085757 +0000 UTC m=+0.328646803,LastTimestamp:2026-07-02 13:43:54.002085757 +0000 UTC m=+0.328646803,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-40.ec2.internal,}" Jul 02 13:43:54.014017 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.013997 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 13:43:54.014128 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.014018 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 13:43:54.014128 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.014112 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:54.014820 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.014645 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 13:43:54.014820 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.014664 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 13:43:54.014820 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.014674 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 13:43:54.014820 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.014769 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 02 13:43:54.014820 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.014777 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 02 13:43:54.015564 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.015548 2561 factory.go:55] Registering systemd factory Jul 02 13:43:54.015630 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.015589 2561 factory.go:223] Registration of the systemd container factory successfully Jul 02 13:43:54.015814 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.015788 2561 factory.go:153] Registering CRI-O factory Jul 02 13:43:54.015814 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.015805 2561 factory.go:223] Registration of the crio container factory successfully Jul 02 13:43:54.015927 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.015858 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 13:43:54.015927 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.015882 2561 factory.go:103] Registering Raw factory Jul 02 13:43:54.015927 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.015895 2561 manager.go:1196] Started watching for new ooms in manager Jul 02 13:43:54.016311 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.016281 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 13:43:54.016402 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.016384 2561 manager.go:349] Starting recovery of all containers Jul 02 13:43:54.016467 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.016437 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 13:43:54.024675 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.024531 2561 manager.go:354] Recovery completed Jul 02 13:43:54.030143 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.030129 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:43:54.031787 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.031769 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:43:54.031841 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.031804 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:43:54.031841 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.031817 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:43:54.032414 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.032399 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 13:43:54.032414 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.032412 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:43:54.032512 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.032414 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 13:43:54.032512 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.032463 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 02 13:43:54.033163 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.033148 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:43:54.033217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.033179 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:43:54.033217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.033193 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:43:54.033694 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.033679 2561 policy_none.go:49] "None policy: Start" Jul 02 13:43:54.033694 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.033695 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 13:43:54.033813 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.033706 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 13:43:54.034782 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.034771 2561 policy_none.go:47] "Start" Jul 02 13:43:54.040151 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.040130 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-crxwz" Jul 02 13:43:54.044480 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.044409 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-40.ec2.internal.18be7ce6fe0b91fd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-40.ec2.internal,UID:ip-10-0-137-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-40.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-40.ec2.internal,},FirstTimestamp:2026-07-02 13:43:54.031788541 +0000 UTC m=+0.358349591,LastTimestamp:2026-07-02 13:43:54.031788541 +0000 UTC m=+0.358349591,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-40.ec2.internal,}" Jul 02 13:43:54.046269 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.046196 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-40.ec2.internal.18be7ce6fe0bebba default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-40.ec2.internal,UID:ip-10-0-137-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-40.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-40.ec2.internal,},FirstTimestamp:2026-07-02 13:43:54.031811514 +0000 UTC m=+0.358372560,LastTimestamp:2026-07-02 13:43:54.031811514 +0000 UTC m=+0.358372560,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-40.ec2.internal,}" Jul 02 13:43:54.047913 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.047899 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-crxwz" Jul 02 13:43:54.079161 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.079133 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 13:43:54.085920 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.080475 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 13:43:54.085920 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.080518 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 13:43:54.085920 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.080543 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 13:43:54.085920 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.080575 2561 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 02 13:43:54.085920 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.082666 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:43:54.115252 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.115118 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:54.125644 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.125627 2561 manager.go:337] "Starting Device Plugin manager" Jul 02 13:43:54.125727 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.125690 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 13:43:54.125727 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.125700 2561 server.go:87] "Starting device plugin registration server" Jul 02 13:43:54.125959 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.125947 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 13:43:54.126005 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.125962 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 13:43:54.126162 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.126146 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 13:43:54.126239 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.126229 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 13:43:54.126282 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.126240 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 13:43:54.126613 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.126583 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 13:43:54.126683 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.126637 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:54.181373 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.181323 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-40.ec2.internal"] Jul 02 13:43:54.181510 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.181430 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:43:54.182409 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.182385 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:43:54.182494 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.182420 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:43:54.182494 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.182434 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:43:54.183745 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.183732 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:43:54.183876 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.183863 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.183933 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.183891 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:43:54.184487 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.184471 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:43:54.184487 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.184482 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:43:54.184604 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.184503 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:43:54.184604 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.184507 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:43:54.184604 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.184517 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:43:54.184604 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.184519 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:43:54.185696 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.185684 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.185746 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.185707 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:43:54.186419 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.186402 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:43:54.186488 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.186428 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:43:54.186488 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.186439 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:43:54.210799 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.210781 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-40.ec2.internal\" not found" node="ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.215409 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.215394 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-40.ec2.internal\" not found" node="ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.222495 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.222477 2561 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-40.ec2.internal\" not found" node="ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.226649 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.226637 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:43:54.227485 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.227467 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:43:54.227574 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.227502 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:43:54.227574 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.227516 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:43:54.227574 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.227545 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.235659 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.235640 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.235659 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.235659 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-40.ec2.internal\": node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:54.269444 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.269384 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:54.315224 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.315199 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cac7e39afd2be0408e453d8c274dd04a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal\" (UID: \"cac7e39afd2be0408e453d8c274dd04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.315314 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.315235 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cac7e39afd2be0408e453d8c274dd04a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal\" (UID: \"cac7e39afd2be0408e453d8c274dd04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.315314 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.315255 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c02ef9b20a4c8c441a824f8fcd9226be-config\") pod \"kube-apiserver-proxy-ip-10-0-137-40.ec2.internal\" (UID: \"c02ef9b20a4c8c441a824f8fcd9226be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.370361 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.370334 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:54.415649 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.415622 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cac7e39afd2be0408e453d8c274dd04a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal\" (UID: \"cac7e39afd2be0408e453d8c274dd04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.415763 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.415659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cac7e39afd2be0408e453d8c274dd04a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal\" (UID: \"cac7e39afd2be0408e453d8c274dd04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.415763 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.415685 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c02ef9b20a4c8c441a824f8fcd9226be-config\") pod \"kube-apiserver-proxy-ip-10-0-137-40.ec2.internal\" (UID: \"c02ef9b20a4c8c441a824f8fcd9226be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.415763 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.415728 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cac7e39afd2be0408e453d8c274dd04a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal\" (UID: \"cac7e39afd2be0408e453d8c274dd04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.415763 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.415737 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c02ef9b20a4c8c441a824f8fcd9226be-config\") pod \"kube-apiserver-proxy-ip-10-0-137-40.ec2.internal\" (UID: \"c02ef9b20a4c8c441a824f8fcd9226be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.415903 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.415728 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cac7e39afd2be0408e453d8c274dd04a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal\" (UID: \"cac7e39afd2be0408e453d8c274dd04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.470895 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.470864 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:54.514227 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.514186 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.518328 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.518313 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-40.ec2.internal" Jul 02 13:43:54.571508 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.571478 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:54.671793 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.671748 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:54.772900 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.772801 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:54.857545 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.857513 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:43:54.874008 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.873976 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:54.958035 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.957996 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 13:43:54.958555 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.958200 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 13:43:54.958555 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:54.958200 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 13:43:54.974437 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:54.974411 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:55.033254 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:55.033181 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcac7e39afd2be0408e453d8c274dd04a.slice/crio-77a9682af8b25a808b1a9b08b19cfd994020f05d03286fbea434a37ccf9d2a3e WatchSource:0}: Error finding container 77a9682af8b25a808b1a9b08b19cfd994020f05d03286fbea434a37ccf9d2a3e: Status 404 returned error can't find the container with id 77a9682af8b25a808b1a9b08b19cfd994020f05d03286fbea434a37ccf9d2a3e Jul 02 13:43:55.037427 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.037402 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 13:43:55.043028 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:55.043006 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc02ef9b20a4c8c441a824f8fcd9226be.slice/crio-6f92d1ec2d9e46446e3b9e18a42c003264c47efa8f92e9db53c6c74602b89371 WatchSource:0}: Error finding container 6f92d1ec2d9e46446e3b9e18a42c003264c47efa8f92e9db53c6c74602b89371: Status 404 returned error can't find the container with id 6f92d1ec2d9e46446e3b9e18a42c003264c47efa8f92e9db53c6c74602b89371 Jul 02 13:43:55.052962 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.052931 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 13:38:54 +0000 UTC" deadline="2028-04-10 23:00:45.356646774 +0000 UTC" Jul 02 13:43:55.052962 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.052961 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15561h16m50.303689229s" Jul 02 13:43:55.074654 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:55.074627 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:55.085009 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.084960 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-40.ec2.internal" event={"ID":"c02ef9b20a4c8c441a824f8fcd9226be","Type":"ContainerStarted","Data":"6f92d1ec2d9e46446e3b9e18a42c003264c47efa8f92e9db53c6c74602b89371"} Jul 02 13:43:55.085952 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.085933 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" event={"ID":"cac7e39afd2be0408e453d8c274dd04a","Type":"ContainerStarted","Data":"77a9682af8b25a808b1a9b08b19cfd994020f05d03286fbea434a37ccf9d2a3e"} Jul 02 13:43:55.140613 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.140592 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:43:55.143081 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.143064 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:43:55.143165 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.143099 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:43:55.143165 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.143112 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:43:55.175533 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:55.175511 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:55.277790 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:55.277746 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-40.ec2.internal\" not found" Jul 02 13:43:55.319494 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.319425 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:43:55.414242 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.414195 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" Jul 02 13:43:55.442299 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.442273 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 13:43:55.443406 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.443391 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-40.ec2.internal" Jul 02 13:43:55.458500 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.458479 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 13:43:55.522397 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.522368 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:43:55.843897 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.843860 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:43:55.993697 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:55.993659 2561 apiserver.go:52] "Watching apiserver" Jul 02 13:43:56.005059 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.005013 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 13:43:56.006518 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.006116 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-lmznz","kube-system/kube-apiserver-proxy-ip-10-0-137-40.ec2.internal","openshift-cluster-node-tuning-operator/tuned-q98ws","openshift-dns/node-resolver-9tw66","openshift-image-registry/node-ca-slrnm","openshift-multus/multus-9vc4r","openshift-multus/multus-additional-cni-plugins-f8xxj","openshift-multus/network-metrics-daemon-p8qsr","kube-system/global-pull-secret-syncer-bj8sb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal","openshift-network-diagnostics/network-check-target-jn2xt","openshift-network-operator/iptables-alerter-rfk79","openshift-ovn-kubernetes/ovnkube-node-x9gx2"] Jul 02 13:43:56.008919 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.008886 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.011506 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.011454 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.012951 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.012929 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 13:43:56.013065 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.012971 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 13:43:56.013065 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.012985 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lztbz\"" Jul 02 13:43:56.013211 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.013156 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 13:43:56.013281 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.013236 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 13:43:56.015060 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.013984 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 13:43:56.015060 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.014347 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:43:56.015060 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.014428 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:43:56.015060 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.015011 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 13:43:56.015392 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.015129 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 13:43:56.015392 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.015140 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f2c66\"" Jul 02 13:43:56.015392 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.015324 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 13:43:56.015392 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.015361 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 13:43:56.015587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.015542 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 13:43:56.015587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.015578 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 13:43:56.016699 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.016443 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9tw66" Jul 02 13:43:56.019278 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.018845 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vrrgz\"" Jul 02 13:43:56.019278 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.018877 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 13:43:56.019278 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.019043 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 13:43:56.019278 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.019224 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.021905 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.021489 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-slrnm" Jul 02 13:43:56.022085 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022064 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdbc7835-070d-4af2-987c-987dd8ab81fd-ovnkube-config\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.022204 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022118 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-cni-dir\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022204 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022147 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-etc-kubernetes\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022204 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022173 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-run-openvswitch\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.022204 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022196 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-node-log\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.022423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022217 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-cni-bin\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.022423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022239 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-cni-netd\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.022423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022252 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 13:43:56.022423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022266 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdbc7835-070d-4af2-987c-987dd8ab81fd-ovnkube-script-lib\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.022423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022305 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-var-lib-cni-bin\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022333 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 13:43:56.022423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022343 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sxmmt\"" Jul 02 13:43:56.022423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022334 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-systemd-units\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022433 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-os-release\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022492 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-run-k8s-cni-cncf-io\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022533 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-run-multus-certs\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022570 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-kubelet\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022616 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-etc-openvswitch\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022650 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-cnibin\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022678 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-socket-dir-parent\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022706 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-run-netns\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022746 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-hostroot\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022783 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmrr9\" (UniqueName: \"kubernetes.io/projected/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-kube-api-access-bmrr9\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022807 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-run-ovn\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022841 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24twn\" (UniqueName: \"kubernetes.io/projected/85ec7384-3925-4a95-a743-fd26e061cb25-kube-api-access-24twn\") pod \"node-resolver-9tw66\" (UID: \"85ec7384-3925-4a95-a743-fd26e061cb25\") " pod="openshift-dns/node-resolver-9tw66" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022864 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-cni-binary-copy\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.022898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022892 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-var-lib-kubelet\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022914 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-daemon-config\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022931 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdbc7835-070d-4af2-987c-987dd8ab81fd-env-overrides\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022946 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdbc7835-070d-4af2-987c-987dd8ab81fd-ovn-node-metrics-cert\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.022968 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85ec7384-3925-4a95-a743-fd26e061cb25-tmp-dir\") pod \"node-resolver-9tw66\" (UID: \"85ec7384-3925-4a95-a743-fd26e061cb25\") " pod="openshift-dns/node-resolver-9tw66" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023007 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-system-cni-dir\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023046 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-var-lib-cni-multus\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023096 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-run-netns\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023152 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-run-systemd\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-var-lib-openvswitch\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023259 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-log-socket\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023289 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023315 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023344 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbdb7\" (UniqueName: \"kubernetes.io/projected/bdbc7835-070d-4af2-987c-987dd8ab81fd-kube-api-access-cbdb7\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023369 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpwcz\" (UniqueName: \"kubernetes.io/projected/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-kube-api-access-hpwcz\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023394 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:43:56.023737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023416 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85ec7384-3925-4a95-a743-fd26e061cb25-hosts-file\") pod \"node-resolver-9tw66\" (UID: \"85ec7384-3925-4a95-a743-fd26e061cb25\") " pod="openshift-dns/node-resolver-9tw66" Jul 02 13:43:56.024583 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023441 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-conf-dir\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.024583 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023475 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-slash\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.024583 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.023919 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 13:43:56.024583 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.024154 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 13:43:56.024583 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.024234 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 13:43:56.024583 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.024240 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4tz9n\"" Jul 02 13:43:56.024974 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.024954 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:43:56.027329 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.027286 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 13:43:56.028001 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.027731 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 13:43:56.028001 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.027740 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 13:43:56.028001 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.027752 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.028001 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.027736 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xv2cj\"" Jul 02 13:43:56.029660 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.029638 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:56.029759 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.029718 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:43:56.030449 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.030101 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 13:43:56.030449 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.030212 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6dxlx\"" Jul 02 13:43:56.030573 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.030484 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 13:43:56.032377 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.032357 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.034720 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.034699 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 13:43:56.035046 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.034863 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pqvtt\"" Jul 02 13:43:56.035177 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.035153 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:43:56.035243 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.035204 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:43:56.035526 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.035390 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 13:43:56.035526 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.035410 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 13:43:56.039822 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.039584 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rfk79" Jul 02 13:43:56.044826 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.044584 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gkj8m\"" Jul 02 13:43:56.044826 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.044616 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 13:43:56.044826 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.044678 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 13:43:56.044826 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.044732 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 13:43:56.065694 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.065667 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s8hvr" Jul 02 13:43:56.072789 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.072765 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s8hvr" Jul 02 13:43:56.115669 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.115589 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 13:43:56.124250 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124223 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-node-log\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.124250 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124254 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-cni-netd\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.124441 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124279 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d4fe52c-af75-465b-906e-3703bc2042ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.124441 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124305 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjkcb\" (UniqueName: \"kubernetes.io/projected/5d4fe52c-af75-465b-906e-3703bc2042ce-kube-api-access-vjkcb\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.124441 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124331 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-socket-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.124441 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124336 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-node-log\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.124441 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124342 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-cni-netd\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.124441 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124381 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-tmp\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.124441 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124432 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-systemd-units\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124462 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-run-k8s-cni-cncf-io\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124487 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-run-multus-certs\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124518 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmwhl\" (UniqueName: \"kubernetes.io/projected/c30d5842-ae96-42d5-8074-d4f321b23818-kube-api-access-tmwhl\") pod \"iptables-alerter-rfk79\" (UID: \"c30d5842-ae96-42d5-8074-d4f321b23818\") " pod="openshift-network-operator/iptables-alerter-rfk79" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124520 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-systemd-units\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124547 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-registration-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124578 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-run-multus-certs\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124584 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-run-k8s-cni-cncf-io\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124577 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-cnibin\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124611 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-run-netns\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124632 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmrr9\" (UniqueName: \"kubernetes.io/projected/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-kube-api-access-bmrr9\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124635 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-cnibin\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124646 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-run-netns\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124672 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-etc-selinux\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124699 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2081cca0-7355-4c02-85dc-b959914fbf47-konnectivity-ca\") pod \"konnectivity-agent-lmznz\" (UID: \"2081cca0-7355-4c02-85dc-b959914fbf47\") " pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124725 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-sysctl-conf\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124781 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-run-ovn\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.124850 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124803 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24twn\" (UniqueName: \"kubernetes.io/projected/85ec7384-3925-4a95-a743-fd26e061cb25-kube-api-access-24twn\") pod \"node-resolver-9tw66\" (UID: \"85ec7384-3925-4a95-a743-fd26e061cb25\") " pod="openshift-dns/node-resolver-9tw66" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124826 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-daemon-config\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124878 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-run-ovn\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124917 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c30d5842-ae96-42d5-8074-d4f321b23818-host-slash\") pod \"iptables-alerter-rfk79\" (UID: \"c30d5842-ae96-42d5-8074-d4f321b23818\") " pod="openshift-network-operator/iptables-alerter-rfk79" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124942 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-host\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124971 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdbc7835-070d-4af2-987c-987dd8ab81fd-env-overrides\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.124997 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85ec7384-3925-4a95-a743-fd26e061cb25-tmp-dir\") pod \"node-resolver-9tw66\" (UID: \"85ec7384-3925-4a95-a743-fd26e061cb25\") " pod="openshift-dns/node-resolver-9tw66" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125038 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-system-cni-dir\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125082 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-var-lib-cni-multus\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125110 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-cnibin\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125137 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-systemd\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125170 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-run-netns\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125196 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-run-systemd\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125223 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-var-lib-openvswitch\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125267 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbdb7\" (UniqueName: \"kubernetes.io/projected/bdbc7835-070d-4af2-987c-987dd8ab81fd-kube-api-access-cbdb7\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125294 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpwcz\" (UniqueName: \"kubernetes.io/projected/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-kube-api-access-hpwcz\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125320 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85ec7384-3925-4a95-a743-fd26e061cb25-hosts-file\") pod \"node-resolver-9tw66\" (UID: \"85ec7384-3925-4a95-a743-fd26e061cb25\") " pod="openshift-dns/node-resolver-9tw66" Jul 02 13:43:56.125711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125345 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-conf-dir\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125371 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wkx9\" (UniqueName: \"kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9\") pod \"network-check-target-jn2xt\" (UID: \"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5\") " pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125416 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85ec7384-3925-4a95-a743-fd26e061cb25-tmp-dir\") pod \"node-resolver-9tw66\" (UID: \"85ec7384-3925-4a95-a743-fd26e061cb25\") " pod="openshift-dns/node-resolver-9tw66" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125419 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c30d5842-ae96-42d5-8074-d4f321b23818-iptables-alerter-script\") pod \"iptables-alerter-rfk79\" (UID: \"c30d5842-ae96-42d5-8074-d4f321b23818\") " pod="openshift-network-operator/iptables-alerter-rfk79" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125492 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-daemon-config\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125532 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdbc7835-070d-4af2-987c-987dd8ab81fd-env-overrides\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125553 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-run-netns\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125577 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-system-cni-dir\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125591 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-run-systemd\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125593 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-var-lib-openvswitch\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125497 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-os-release\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125624 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-var-lib-cni-multus\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125651 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-sys-fs\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125678 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmmbt\" (UniqueName: \"kubernetes.io/projected/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-kube-api-access-jmmbt\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125681 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-conf-dir\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125704 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-run\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125683 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85ec7384-3925-4a95-a743-fd26e061cb25-hosts-file\") pod \"node-resolver-9tw66\" (UID: \"85ec7384-3925-4a95-a743-fd26e061cb25\") " pod="openshift-dns/node-resolver-9tw66" Jul 02 13:43:56.126540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125732 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-lib-modules\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125757 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-var-lib-kubelet\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125781 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-tuned\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125838 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-slash\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125890 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-slash\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125895 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdbc7835-070d-4af2-987c-987dd8ab81fd-ovnkube-config\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125929 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-etc-kubernetes\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-system-cni-dir\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125980 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-sysconfig\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.125997 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-run-openvswitch\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126014 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-cni-bin\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126040 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdbc7835-070d-4af2-987c-987dd8ab81fd-ovnkube-script-lib\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126076 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-etc-kubernetes\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126115 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-var-lib-cni-bin\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126117 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-cni-bin\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126143 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-kubernetes\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126175 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-run-openvswitch\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.127341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126239 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-os-release\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126260 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-var-lib-cni-bin\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126288 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-kubelet\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126306 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-etc-openvswitch\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126342 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-socket-dir-parent\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126368 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-hostroot\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126381 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-os-release\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126402 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-etc-openvswitch\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126427 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdbc7835-070d-4af2-987c-987dd8ab81fd-ovnkube-config\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126463 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-kubelet\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126476 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-hostroot\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126475 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d4fe52c-af75-465b-906e-3703bc2042ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126511 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4e02fa4-8728-4b3e-a301-fb8038dc5147-host\") pod \"node-ca-slrnm\" (UID: \"a4e02fa4-8728-4b3e-a301-fb8038dc5147\") " pod="openshift-image-registry/node-ca-slrnm" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126554 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-cni-binary-copy\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126585 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-socket-dir-parent\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126584 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-var-lib-kubelet\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126619 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-device-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126624 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdbc7835-070d-4af2-987c-987dd8ab81fd-ovnkube-script-lib\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.128216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126637 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-modprobe-d\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126656 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdbc7835-070d-4af2-987c-987dd8ab81fd-ovn-node-metrics-cert\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126674 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d4fe52c-af75-465b-906e-3703bc2042ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126691 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqsds\" (UniqueName: \"kubernetes.io/projected/a4e02fa4-8728-4b3e-a301-fb8038dc5147-kube-api-access-hqsds\") pod \"node-ca-slrnm\" (UID: \"a4e02fa4-8728-4b3e-a301-fb8038dc5147\") " pod="openshift-image-registry/node-ca-slrnm" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126713 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-log-socket\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126735 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-host-var-lib-kubelet\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126762 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-log-socket\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126811 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126840 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126869 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126912 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126935 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdbc7835-070d-4af2-987c-987dd8ab81fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126947 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.126989 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2081cca0-7355-4c02-85dc-b959914fbf47-agent-certs\") pod \"konnectivity-agent-lmznz\" (UID: \"2081cca0-7355-4c02-85dc-b959914fbf47\") " pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127007 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-cni-binary-copy\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127025 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-dbus\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127023 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 13:43:56.129108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127047 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:56.129949 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127078 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:43:56.129949 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127103 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a4e02fa4-8728-4b3e-a301-fb8038dc5147-serviceca\") pod \"node-ca-slrnm\" (UID: \"a4e02fa4-8728-4b3e-a301-fb8038dc5147\") " pod="openshift-image-registry/node-ca-slrnm" Jul 02 13:43:56.129949 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127135 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-sysctl-d\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.129949 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127161 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzm4b\" (UniqueName: \"kubernetes.io/projected/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-kube-api-access-bzm4b\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.129949 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.127173 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:43:56.129949 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127190 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-kubelet-config\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:56.129949 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.127248 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs podName:48976aeb-7f1a-4164-a77b-9bdc07d79cb9 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:56.627231956 +0000 UTC m=+2.953793021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs") pod "network-metrics-daemon-p8qsr" (UID: "48976aeb-7f1a-4164-a77b-9bdc07d79cb9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:43:56.129949 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127265 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-cni-dir\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.129949 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127290 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-sys\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.129949 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.127387 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-multus-cni-dir\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.130474 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.130414 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdbc7835-070d-4af2-987c-987dd8ab81fd-ovn-node-metrics-cert\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.136734 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.136708 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpwcz\" (UniqueName: \"kubernetes.io/projected/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-kube-api-access-hpwcz\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:43:56.136993 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.136974 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24twn\" (UniqueName: \"kubernetes.io/projected/85ec7384-3925-4a95-a743-fd26e061cb25-kube-api-access-24twn\") pod \"node-resolver-9tw66\" (UID: \"85ec7384-3925-4a95-a743-fd26e061cb25\") " pod="openshift-dns/node-resolver-9tw66" Jul 02 13:43:56.137344 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.137319 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbdb7\" (UniqueName: \"kubernetes.io/projected/bdbc7835-070d-4af2-987c-987dd8ab81fd-kube-api-access-cbdb7\") pod \"ovnkube-node-x9gx2\" (UID: \"bdbc7835-070d-4af2-987c-987dd8ab81fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.137875 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.137853 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmrr9\" (UniqueName: \"kubernetes.io/projected/d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3-kube-api-access-bmrr9\") pod \"multus-9vc4r\" (UID: \"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3\") " pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.228169 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228119 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-var-lib-kubelet\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.228169 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228161 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-tuned\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.228423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228192 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-system-cni-dir\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.228423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228216 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-sysconfig\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.228423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228241 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-kubernetes\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.228423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228253 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-var-lib-kubelet\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.228423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228330 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-system-cni-dir\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.228423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228338 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-sysconfig\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.228423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228355 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d4fe52c-af75-465b-906e-3703bc2042ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.228423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228388 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4e02fa4-8728-4b3e-a301-fb8038dc5147-host\") pod \"node-ca-slrnm\" (UID: \"a4e02fa4-8728-4b3e-a301-fb8038dc5147\") " pod="openshift-image-registry/node-ca-slrnm" Jul 02 13:43:56.228423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228391 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-kubernetes\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.228423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-device-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228445 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-modprobe-d\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228458 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-device-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228484 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4e02fa4-8728-4b3e-a301-fb8038dc5147-host\") pod \"node-ca-slrnm\" (UID: \"a4e02fa4-8728-4b3e-a301-fb8038dc5147\") " pod="openshift-image-registry/node-ca-slrnm" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228484 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d4fe52c-af75-465b-906e-3703bc2042ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228540 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqsds\" (UniqueName: \"kubernetes.io/projected/a4e02fa4-8728-4b3e-a301-fb8038dc5147-kube-api-access-hqsds\") pod \"node-ca-slrnm\" (UID: \"a4e02fa4-8728-4b3e-a301-fb8038dc5147\") " pod="openshift-image-registry/node-ca-slrnm" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228580 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-modprobe-d\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228598 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228627 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2081cca0-7355-4c02-85dc-b959914fbf47-agent-certs\") pod \"konnectivity-agent-lmznz\" (UID: \"2081cca0-7355-4c02-85dc-b959914fbf47\") " pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228694 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-dbus\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228719 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228757 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a4e02fa4-8728-4b3e-a301-fb8038dc5147-serviceca\") pod \"node-ca-slrnm\" (UID: \"a4e02fa4-8728-4b3e-a301-fb8038dc5147\") " pod="openshift-image-registry/node-ca-slrnm" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228784 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-sysctl-d\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228781 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228811 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzm4b\" (UniqueName: \"kubernetes.io/projected/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-kube-api-access-bzm4b\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228855 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-kubelet-config\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:56.228935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228890 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-sys\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228951 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228953 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-sys\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228969 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d4fe52c-af75-465b-906e-3703bc2042ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228992 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d4fe52c-af75-465b-906e-3703bc2042ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.228994 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-kubelet-config\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229020 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d4fe52c-af75-465b-906e-3703bc2042ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229022 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkcb\" (UniqueName: \"kubernetes.io/projected/5d4fe52c-af75-465b-906e-3703bc2042ce-kube-api-access-vjkcb\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229062 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-socket-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.229096 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229095 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-tmp\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229163 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmwhl\" (UniqueName: \"kubernetes.io/projected/c30d5842-ae96-42d5-8074-d4f321b23818-kube-api-access-tmwhl\") pod \"iptables-alerter-rfk79\" (UID: \"c30d5842-ae96-42d5-8074-d4f321b23818\") " pod="openshift-network-operator/iptables-alerter-rfk79" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.229262 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret podName:c0b8e79b-9ee6-4442-bb4b-5020ff3833a9 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:56.72916343 +0000 UTC m=+3.055724465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret") pod "global-pull-secret-syncer-bj8sb" (UID: "c0b8e79b-9ee6-4442-bb4b-5020ff3833a9") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229308 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-registration-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229346 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-etc-selinux\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229392 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2081cca0-7355-4c02-85dc-b959914fbf47-konnectivity-ca\") pod \"konnectivity-agent-lmznz\" (UID: \"2081cca0-7355-4c02-85dc-b959914fbf47\") " pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:43:56.229830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229418 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-sysctl-conf\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.230681 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229455 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-registration-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.230681 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229892 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-socket-dir\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.230681 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.230209 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-etc-selinux\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.230681 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.229450 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c30d5842-ae96-42d5-8074-d4f321b23818-host-slash\") pod \"iptables-alerter-rfk79\" (UID: \"c30d5842-ae96-42d5-8074-d4f321b23818\") " pod="openshift-network-operator/iptables-alerter-rfk79" Jul 02 13:43:56.230681 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.230316 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-host\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.230681 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.230405 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-cnibin\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.230681 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.230456 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-host\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.230681 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.230488 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d4fe52c-af75-465b-906e-3703bc2042ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.230681 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.230643 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c30d5842-ae96-42d5-8074-d4f321b23818-host-slash\") pod \"iptables-alerter-rfk79\" (UID: \"c30d5842-ae96-42d5-8074-d4f321b23818\") " pod="openshift-network-operator/iptables-alerter-rfk79" Jul 02 13:43:56.231178 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.230993 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2081cca0-7355-4c02-85dc-b959914fbf47-konnectivity-ca\") pod \"konnectivity-agent-lmznz\" (UID: \"2081cca0-7355-4c02-85dc-b959914fbf47\") " pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:43:56.231178 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231074 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-dbus\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:56.231287 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231181 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-tuned\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.231287 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231256 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-sysctl-conf\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.231459 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231418 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-sysctl-d\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.231554 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.230356 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-cnibin\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.231609 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231569 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-systemd\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.231666 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231629 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkx9\" (UniqueName: \"kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9\") pod \"network-check-target-jn2xt\" (UID: \"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5\") " pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231756 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a4e02fa4-8728-4b3e-a301-fb8038dc5147-serviceca\") pod \"node-ca-slrnm\" (UID: \"a4e02fa4-8728-4b3e-a301-fb8038dc5147\") " pod="openshift-image-registry/node-ca-slrnm" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231766 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c30d5842-ae96-42d5-8074-d4f321b23818-iptables-alerter-script\") pod \"iptables-alerter-rfk79\" (UID: \"c30d5842-ae96-42d5-8074-d4f321b23818\") " pod="openshift-network-operator/iptables-alerter-rfk79" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231891 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2081cca0-7355-4c02-85dc-b959914fbf47-agent-certs\") pod \"konnectivity-agent-lmznz\" (UID: \"2081cca0-7355-4c02-85dc-b959914fbf47\") " pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231907 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-os-release\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231942 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-sys-fs\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231973 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmmbt\" (UniqueName: \"kubernetes.io/projected/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-kube-api-access-jmmbt\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.231999 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d4fe52c-af75-465b-906e-3703bc2042ce-os-release\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.232002 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-run\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.232081 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-run\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.232093 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-lib-modules\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.232144 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-sys-fs\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.232258 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-etc-systemd\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.232294 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-lib-modules\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.234267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.232409 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c30d5842-ae96-42d5-8074-d4f321b23818-iptables-alerter-script\") pod \"iptables-alerter-rfk79\" (UID: \"c30d5842-ae96-42d5-8074-d4f321b23818\") " pod="openshift-network-operator/iptables-alerter-rfk79" Jul 02 13:43:56.236238 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.236212 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-tmp\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.242442 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.242416 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqsds\" (UniqueName: \"kubernetes.io/projected/a4e02fa4-8728-4b3e-a301-fb8038dc5147-kube-api-access-hqsds\") pod \"node-ca-slrnm\" (UID: \"a4e02fa4-8728-4b3e-a301-fb8038dc5147\") " pod="openshift-image-registry/node-ca-slrnm" Jul 02 13:43:56.247691 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.247670 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzm4b\" (UniqueName: \"kubernetes.io/projected/caa799d4-3d86-4b19-9af0-2d4b19f82cb4-kube-api-access-bzm4b\") pod \"tuned-q98ws\" (UID: \"caa799d4-3d86-4b19-9af0-2d4b19f82cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.250160 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.250109 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjkcb\" (UniqueName: \"kubernetes.io/projected/5d4fe52c-af75-465b-906e-3703bc2042ce-kube-api-access-vjkcb\") pod \"multus-additional-cni-plugins-f8xxj\" (UID: \"5d4fe52c-af75-465b-906e-3703bc2042ce\") " pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.250590 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.250568 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:43:56.250698 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.250593 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:43:56.250698 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.250608 2561 projected.go:196] Error preparing data for projected volume kube-api-access-8wkx9 for pod openshift-network-diagnostics/network-check-target-jn2xt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:43:56.250698 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.250696 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9 podName:4ef79fd6-3648-4b4b-bd70-38b27b45aeb5 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:56.750675947 +0000 UTC m=+3.077237012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8wkx9" (UniqueName: "kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9") pod "network-check-target-jn2xt" (UID: "4ef79fd6-3648-4b4b-bd70-38b27b45aeb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:43:56.253332 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.253300 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmmbt\" (UniqueName: \"kubernetes.io/projected/03a9d000-12a0-4b05-9e8d-0c89d22e59c9-kube-api-access-jmmbt\") pod \"aws-ebs-csi-driver-node-pxd44\" (UID: \"03a9d000-12a0-4b05-9e8d-0c89d22e59c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.253431 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.253370 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmwhl\" (UniqueName: \"kubernetes.io/projected/c30d5842-ae96-42d5-8074-d4f321b23818-kube-api-access-tmwhl\") pod \"iptables-alerter-rfk79\" (UID: \"c30d5842-ae96-42d5-8074-d4f321b23818\") " pod="openshift-network-operator/iptables-alerter-rfk79" Jul 02 13:43:56.324719 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.324684 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9vc4r" Jul 02 13:43:56.333351 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.333327 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:43:56.346546 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.346516 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9tw66" Jul 02 13:43:56.351677 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.351656 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" Jul 02 13:43:56.360470 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.360435 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-slrnm" Jul 02 13:43:56.370527 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.370438 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:43:56.378488 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.378461 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q98ws" Jul 02 13:43:56.387443 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.387417 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" Jul 02 13:43:56.395312 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.395290 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rfk79" Jul 02 13:43:56.635498 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.635405 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:43:56.635677 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.635609 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:43:56.635731 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.635688 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs podName:48976aeb-7f1a-4164-a77b-9bdc07d79cb9 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:57.635669677 +0000 UTC m=+3.962230718 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs") pod "network-metrics-daemon-p8qsr" (UID: "48976aeb-7f1a-4164-a77b-9bdc07d79cb9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:43:56.736536 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.736490 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:56.736718 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.736674 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:43:56.736793 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.736741 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret podName:c0b8e79b-9ee6-4442-bb4b-5020ff3833a9 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:57.736723681 +0000 UTC m=+4.063284721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret") pod "global-pull-secret-syncer-bj8sb" (UID: "c0b8e79b-9ee6-4442-bb4b-5020ff3833a9") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:43:56.837494 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:56.837454 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkx9\" (UniqueName: \"kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9\") pod \"network-check-target-jn2xt\" (UID: \"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5\") " pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:43:56.837663 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.837624 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:43:56.837663 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.837645 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:43:56.837663 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.837660 2561 projected.go:196] Error preparing data for projected volume kube-api-access-8wkx9 for pod openshift-network-diagnostics/network-check-target-jn2xt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:43:56.837818 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:56.837730 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9 podName:4ef79fd6-3648-4b4b-bd70-38b27b45aeb5 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:57.837710413 +0000 UTC m=+4.164271468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wkx9" (UniqueName: "kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9") pod "network-check-target-jn2xt" (UID: "4ef79fd6-3648-4b4b-bd70-38b27b45aeb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:43:57.076818 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:57.076696 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 13:38:56 +0000 UTC" deadline="2028-03-11 08:35:33.411074138 +0000 UTC" Jul 02 13:43:57.076818 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:57.076759 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14826h51m36.334320297s" Jul 02 13:43:57.081069 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:57.081032 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:43:57.081277 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:57.081203 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:43:57.202361 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:57.202328 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ec7384_3925_4a95_a743_fd26e061cb25.slice/crio-2c47e7ad67db1772b99c3b96afdeed869a7cc3340fa61ae3fc1dc4ba86ca714a WatchSource:0}: Error finding container 2c47e7ad67db1772b99c3b96afdeed869a7cc3340fa61ae3fc1dc4ba86ca714a: Status 404 returned error can't find the container with id 2c47e7ad67db1772b99c3b96afdeed869a7cc3340fa61ae3fc1dc4ba86ca714a Jul 02 13:43:57.205186 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:57.205159 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30d5842_ae96_42d5_8074_d4f321b23818.slice/crio-375a8faade2ef3dfda1e2f64c8edc4e34e4bb79851a6ee708df288440db41625 WatchSource:0}: Error finding container 375a8faade2ef3dfda1e2f64c8edc4e34e4bb79851a6ee708df288440db41625: Status 404 returned error can't find the container with id 375a8faade2ef3dfda1e2f64c8edc4e34e4bb79851a6ee708df288440db41625 Jul 02 13:43:57.206067 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:57.206000 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdbc7835_070d_4af2_987c_987dd8ab81fd.slice/crio-c2eae1dee8a0a56e3fa548fabbc3b8b200dae532bac3e3599da2d2a1e0e1025a WatchSource:0}: Error finding container c2eae1dee8a0a56e3fa548fabbc3b8b200dae532bac3e3599da2d2a1e0e1025a: Status 404 returned error can't find the container with id c2eae1dee8a0a56e3fa548fabbc3b8b200dae532bac3e3599da2d2a1e0e1025a Jul 02 13:43:57.208401 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:57.208367 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d4fe52c_af75_465b_906e_3703bc2042ce.slice/crio-2c0b11db339288dca036bf22723c3c13bf2c8ce9c96dd0427b2ab1a33722cb6e WatchSource:0}: Error finding container 2c0b11db339288dca036bf22723c3c13bf2c8ce9c96dd0427b2ab1a33722cb6e: Status 404 returned error can't find the container with id 2c0b11db339288dca036bf22723c3c13bf2c8ce9c96dd0427b2ab1a33722cb6e Jul 02 13:43:57.209126 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:57.209093 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2081cca0_7355_4c02_85dc_b959914fbf47.slice/crio-58ce1b75dd31bfed2508eae497061e130af1395a411ec77a2146fa49b7ee8087 WatchSource:0}: Error finding container 58ce1b75dd31bfed2508eae497061e130af1395a411ec77a2146fa49b7ee8087: Status 404 returned error can't find the container with id 58ce1b75dd31bfed2508eae497061e130af1395a411ec77a2146fa49b7ee8087 Jul 02 13:43:57.211300 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:57.210551 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaa799d4_3d86_4b19_9af0_2d4b19f82cb4.slice/crio-0ae6c43411af86fd49d4d56bd60d5e20df5b01dbe0e75f03ad2bc864aad0d545 WatchSource:0}: Error finding container 0ae6c43411af86fd49d4d56bd60d5e20df5b01dbe0e75f03ad2bc864aad0d545: Status 404 returned error can't find the container with id 0ae6c43411af86fd49d4d56bd60d5e20df5b01dbe0e75f03ad2bc864aad0d545 Jul 02 13:43:57.216489 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:43:57.216466 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ae3d5a_ec79_41a6_8ae7_fb54578f26d3.slice/crio-05d4a19b344e83e85be9a17c41a6eb6ce23c4cc0152d866cc3060f7f4a802a06 WatchSource:0}: Error finding container 05d4a19b344e83e85be9a17c41a6eb6ce23c4cc0152d866cc3060f7f4a802a06: Status 404 returned error can't find the container with id 05d4a19b344e83e85be9a17c41a6eb6ce23c4cc0152d866cc3060f7f4a802a06 Jul 02 13:43:57.644026 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:57.643752 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:43:57.644247 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:57.644037 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:43:57.644247 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:57.644125 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs podName:48976aeb-7f1a-4164-a77b-9bdc07d79cb9 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:59.64410403 +0000 UTC m=+5.970665076 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs") pod "network-metrics-daemon-p8qsr" (UID: "48976aeb-7f1a-4164-a77b-9bdc07d79cb9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:43:57.745921 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:57.745199 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:57.745921 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:57.745403 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:43:57.745921 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:57.745475 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret podName:c0b8e79b-9ee6-4442-bb4b-5020ff3833a9 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:59.745455942 +0000 UTC m=+6.072016981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret") pod "global-pull-secret-syncer-bj8sb" (UID: "c0b8e79b-9ee6-4442-bb4b-5020ff3833a9") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:43:57.823717 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:57.821038 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:43:57.846647 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:57.845895 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkx9\" (UniqueName: \"kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9\") pod \"network-check-target-jn2xt\" (UID: \"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5\") " pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:43:57.846647 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:57.846135 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:43:57.846647 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:57.846162 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:43:57.846647 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:57.846175 2561 projected.go:196] Error preparing data for projected volume kube-api-access-8wkx9 for pod openshift-network-diagnostics/network-check-target-jn2xt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:43:57.846647 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:57.846238 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9 podName:4ef79fd6-3648-4b4b-bd70-38b27b45aeb5 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:59.846217703 +0000 UTC m=+6.172778744 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wkx9" (UniqueName: "kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9") pod "network-check-target-jn2xt" (UID: "4ef79fd6-3648-4b4b-bd70-38b27b45aeb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:43:58.079539 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.077775 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 13:38:56 +0000 UTC" deadline="2028-02-28 05:48:40.930499591 +0000 UTC" Jul 02 13:43:58.079539 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.077846 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14536h4m42.85265734s" Jul 02 13:43:58.086963 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.086927 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:58.087099 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:58.087071 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:43:58.087556 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.087536 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:43:58.087680 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:58.087659 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:43:58.109210 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.109171 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-40.ec2.internal" event={"ID":"c02ef9b20a4c8c441a824f8fcd9226be","Type":"ContainerStarted","Data":"48e38cb87ea42a7130c526a7eaaf890f9a90f48cd71b6092a0863b63d41ac592"} Jul 02 13:43:58.115284 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.115251 2561 generic.go:356] "Generic (PLEG): container finished" podID="cac7e39afd2be0408e453d8c274dd04a" containerID="aca97720a1977d30fb77e05ed01365302dfddcd92ddbf95b285d7f628c9b9708" exitCode=0 Jul 02 13:43:58.115571 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.115478 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" event={"ID":"cac7e39afd2be0408e453d8c274dd04a","Type":"ContainerDied","Data":"aca97720a1977d30fb77e05ed01365302dfddcd92ddbf95b285d7f628c9b9708"} Jul 02 13:43:58.126766 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.126735 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-slrnm" event={"ID":"a4e02fa4-8728-4b3e-a301-fb8038dc5147","Type":"ContainerStarted","Data":"d65cec8aab16200a4c80b0e0430ed409c197b17b63ec1c4278700eec6d885268"} Jul 02 13:43:58.128354 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.128294 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-40.ec2.internal" podStartSLOduration=3.128275097 podStartE2EDuration="3.128275097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:58.127661337 +0000 UTC m=+4.454222398" watchObservedRunningTime="2026-07-02 13:43:58.128275097 +0000 UTC m=+4.454836158" Jul 02 13:43:58.137329 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.136940 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9vc4r" event={"ID":"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3","Type":"ContainerStarted","Data":"05d4a19b344e83e85be9a17c41a6eb6ce23c4cc0152d866cc3060f7f4a802a06"} Jul 02 13:43:58.162682 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.159887 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" event={"ID":"03a9d000-12a0-4b05-9e8d-0c89d22e59c9","Type":"ContainerStarted","Data":"e649a7f944928e4da918f28c22e88294b186441bb5b3f4bfa3a08d8840d8452a"} Jul 02 13:43:58.171130 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.171067 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" event={"ID":"5d4fe52c-af75-465b-906e-3703bc2042ce","Type":"ContainerStarted","Data":"2c0b11db339288dca036bf22723c3c13bf2c8ce9c96dd0427b2ab1a33722cb6e"} Jul 02 13:43:58.190545 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.190413 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lmznz" event={"ID":"2081cca0-7355-4c02-85dc-b959914fbf47","Type":"ContainerStarted","Data":"58ce1b75dd31bfed2508eae497061e130af1395a411ec77a2146fa49b7ee8087"} Jul 02 13:43:58.201158 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.201080 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rfk79" event={"ID":"c30d5842-ae96-42d5-8074-d4f321b23818","Type":"ContainerStarted","Data":"375a8faade2ef3dfda1e2f64c8edc4e34e4bb79851a6ee708df288440db41625"} Jul 02 13:43:58.219382 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.219319 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q98ws" event={"ID":"caa799d4-3d86-4b19-9af0-2d4b19f82cb4","Type":"ContainerStarted","Data":"0ae6c43411af86fd49d4d56bd60d5e20df5b01dbe0e75f03ad2bc864aad0d545"} Jul 02 13:43:58.256225 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.256060 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" event={"ID":"bdbc7835-070d-4af2-987c-987dd8ab81fd","Type":"ContainerStarted","Data":"c2eae1dee8a0a56e3fa548fabbc3b8b200dae532bac3e3599da2d2a1e0e1025a"} Jul 02 13:43:58.279432 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:58.279314 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9tw66" event={"ID":"85ec7384-3925-4a95-a743-fd26e061cb25","Type":"ContainerStarted","Data":"2c47e7ad67db1772b99c3b96afdeed869a7cc3340fa61ae3fc1dc4ba86ca714a"} Jul 02 13:43:59.081460 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:59.081374 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:43:59.081958 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:59.081515 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:43:59.300813 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:59.300776 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" event={"ID":"cac7e39afd2be0408e453d8c274dd04a","Type":"ContainerStarted","Data":"7a841ca0fac862d930a9d9cf4601966cdebd7d8423cd52274584156b27c96727"} Jul 02 13:43:59.663845 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:59.663806 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:43:59.664050 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:59.663967 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:43:59.664050 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:59.664042 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs podName:48976aeb-7f1a-4164-a77b-9bdc07d79cb9 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:03.664021564 +0000 UTC m=+9.990582613 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs") pod "network-metrics-daemon-p8qsr" (UID: "48976aeb-7f1a-4164-a77b-9bdc07d79cb9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:43:59.765910 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:59.765113 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:43:59.765910 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:59.765333 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:43:59.765910 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:59.765443 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret podName:c0b8e79b-9ee6-4442-bb4b-5020ff3833a9 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:03.765423255 +0000 UTC m=+10.091984294 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret") pod "global-pull-secret-syncer-bj8sb" (UID: "c0b8e79b-9ee6-4442-bb4b-5020ff3833a9") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:43:59.866048 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:43:59.865965 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkx9\" (UniqueName: \"kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9\") pod \"network-check-target-jn2xt\" (UID: \"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5\") " pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:43:59.866259 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:59.866190 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:43:59.866259 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:59.866219 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:43:59.866259 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:59.866238 2561 projected.go:196] Error preparing data for projected volume kube-api-access-8wkx9 for pod openshift-network-diagnostics/network-check-target-jn2xt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:43:59.866440 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:43:59.866314 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9 podName:4ef79fd6-3648-4b4b-bd70-38b27b45aeb5 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:03.866293948 +0000 UTC m=+10.192855010 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wkx9" (UniqueName: "kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9") pod "network-check-target-jn2xt" (UID: "4ef79fd6-3648-4b4b-bd70-38b27b45aeb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:44:00.084209 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:00.083929 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:00.084209 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:00.084045 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:00.084209 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:00.084089 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:00.084802 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:00.084221 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:01.081614 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:01.081579 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:01.081828 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:01.081730 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:02.098575 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:02.098539 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:02.100165 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:02.099768 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:02.100165 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:02.100067 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:02.100371 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:02.100261 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:03.081759 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:03.081588 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:03.081759 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:03.081698 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:03.714842 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:03.714260 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:03.714842 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:03.714430 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:44:03.714842 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:03.714492 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs podName:48976aeb-7f1a-4164-a77b-9bdc07d79cb9 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:11.714473323 +0000 UTC m=+18.041034359 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs") pod "network-metrics-daemon-p8qsr" (UID: "48976aeb-7f1a-4164-a77b-9bdc07d79cb9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:44:03.814858 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:03.814817 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:03.815092 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:03.814978 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:44:03.815092 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:03.815074 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret podName:c0b8e79b-9ee6-4442-bb4b-5020ff3833a9 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:11.815052579 +0000 UTC m=+18.141613627 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret") pod "global-pull-secret-syncer-bj8sb" (UID: "c0b8e79b-9ee6-4442-bb4b-5020ff3833a9") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:44:03.916623 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:03.916014 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkx9\" (UniqueName: \"kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9\") pod \"network-check-target-jn2xt\" (UID: \"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5\") " pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:03.916623 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:03.916196 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:44:03.916623 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:03.916219 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:44:03.916623 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:03.916232 2561 projected.go:196] Error preparing data for projected volume kube-api-access-8wkx9 for pod openshift-network-diagnostics/network-check-target-jn2xt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:44:03.916623 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:03.916289 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9 podName:4ef79fd6-3648-4b4b-bd70-38b27b45aeb5 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:11.916272231 +0000 UTC m=+18.242833282 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wkx9" (UniqueName: "kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9") pod "network-check-target-jn2xt" (UID: "4ef79fd6-3648-4b4b-bd70-38b27b45aeb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:44:04.084460 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:04.084337 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:04.084681 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:04.084467 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:04.084840 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:04.084819 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:04.084944 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:04.084921 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:05.081702 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:05.081667 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:05.082192 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:05.081806 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:06.086001 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:06.085538 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:06.086001 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:06.085678 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:06.086001 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:06.085728 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:06.086001 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:06.085841 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:07.081288 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:07.081257 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:07.081462 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:07.081377 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:08.081663 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:08.081638 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:08.082031 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:08.081735 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:08.082031 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:08.081794 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:08.082031 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:08.081883 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:09.081556 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:09.081517 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:09.081746 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:09.081648 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:10.081193 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:10.081154 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:10.081193 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:10.081205 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:10.081446 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:10.081319 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:10.081516 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:10.081440 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:11.080913 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:11.080877 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:11.081469 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:11.081004 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:11.778026 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:11.777978 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:11.778212 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:11.778169 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:44:11.778288 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:11.778247 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs podName:48976aeb-7f1a-4164-a77b-9bdc07d79cb9 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:27.778226538 +0000 UTC m=+34.104787572 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs") pod "network-metrics-daemon-p8qsr" (UID: "48976aeb-7f1a-4164-a77b-9bdc07d79cb9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:44:11.878606 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:11.878572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:11.878776 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:11.878752 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:44:11.878846 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:11.878834 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret podName:c0b8e79b-9ee6-4442-bb4b-5020ff3833a9 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:27.878812628 +0000 UTC m=+34.205373664 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret") pod "global-pull-secret-syncer-bj8sb" (UID: "c0b8e79b-9ee6-4442-bb4b-5020ff3833a9") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:44:11.979101 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:11.979059 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkx9\" (UniqueName: \"kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9\") pod \"network-check-target-jn2xt\" (UID: \"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5\") " pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:11.979267 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:11.979231 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:44:11.979267 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:11.979249 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:44:11.979267 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:11.979259 2561 projected.go:196] Error preparing data for projected volume kube-api-access-8wkx9 for pod openshift-network-diagnostics/network-check-target-jn2xt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:44:11.979387 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:11.979311 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9 podName:4ef79fd6-3648-4b4b-bd70-38b27b45aeb5 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:27.979297781 +0000 UTC m=+34.305858817 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wkx9" (UniqueName: "kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9") pod "network-check-target-jn2xt" (UID: "4ef79fd6-3648-4b4b-bd70-38b27b45aeb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:44:12.081361 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:12.081279 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:12.081361 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:12.081330 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:12.081863 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:12.081427 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:12.081863 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:12.081577 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:13.081011 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:13.080979 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:13.081190 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:13.081093 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:14.081991 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:14.081949 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:14.082452 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:14.082055 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:14.082452 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:14.082093 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:14.082452 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:14.082118 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:15.081208 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:15.081174 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:15.081379 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:15.081283 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:16.083594 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.083274 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:16.083594 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:16.083406 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:16.083594 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.083475 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:16.083594 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:16.083549 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:16.340482 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.340244 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-slrnm" event={"ID":"a4e02fa4-8728-4b3e-a301-fb8038dc5147","Type":"ContainerStarted","Data":"4133f8a301497fa87ccdfa2a1a28bbff10a80112625831ea1d9bffa2d73ae74d"} Jul 02 13:44:16.341926 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.341900 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9vc4r" event={"ID":"d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3","Type":"ContainerStarted","Data":"5213b7ea21b9bc319a3563fa49002bea82eaac6501108ad3a2ce372387c60d50"} Jul 02 13:44:16.343150 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.343133 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" event={"ID":"03a9d000-12a0-4b05-9e8d-0c89d22e59c9","Type":"ContainerStarted","Data":"74b46a0a97d5a4663c34cc60661c94bfa90b39d8c82b259987ed7137361a8887"} Jul 02 13:44:16.351131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.344987 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" event={"ID":"5d4fe52c-af75-465b-906e-3703bc2042ce","Type":"ContainerStarted","Data":"5126872d96daba1f608953355d154038c2b9ad8ebc325b4b265f711cdd34a12f"} Jul 02 13:44:16.351131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.346851 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lmznz" event={"ID":"2081cca0-7355-4c02-85dc-b959914fbf47","Type":"ContainerStarted","Data":"0e183e4b8809f252e6107bbe5091a2a9f56e208f7ed37d3eb92e88ebf88d397c"} Jul 02 13:44:16.351131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.348125 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q98ws" event={"ID":"caa799d4-3d86-4b19-9af0-2d4b19f82cb4","Type":"ContainerStarted","Data":"e4d26e0b59b65fa2325d32b3e175f94ac8cfc6342d7188b4e990776d42b9131d"} Jul 02 13:44:16.351131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.349538 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" event={"ID":"bdbc7835-070d-4af2-987c-987dd8ab81fd","Type":"ContainerStarted","Data":"ac7f11e50155d8d21e3f1916bd1f150b0ff6e1e2fb7c246d708a472046555f23"} Jul 02 13:44:16.351131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.349584 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" event={"ID":"bdbc7835-070d-4af2-987c-987dd8ab81fd","Type":"ContainerStarted","Data":"2de134c425a870329005bde118fb6941af621838759c86172800a26fa1f9511d"} Jul 02 13:44:16.351131 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.350661 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9tw66" event={"ID":"85ec7384-3925-4a95-a743-fd26e061cb25","Type":"ContainerStarted","Data":"047b2fc16c5629df31f8cb05054836b3582898d80bc4af5d4629038b899ae6e5"} Jul 02 13:44:16.359278 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.359242 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-slrnm" podStartSLOduration=2.532596213 podStartE2EDuration="21.359230828s" totalImagesPullingTime="18.826634615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:16.359229728 +0000 UTC m=+22.685790785" watchObservedRunningTime="2026-07-02 13:44:16.359230828 +0000 UTC m=+22.685791882" Jul 02 13:44:16.359738 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.359693 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-40.ec2.internal" podStartSLOduration=21.359682463 podStartE2EDuration="21.359682463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:59.333000301 +0000 UTC m=+5.659561360" watchObservedRunningTime="2026-07-02 13:44:16.359682463 +0000 UTC m=+22.686243518" Jul 02 13:44:16.379702 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.379652 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9vc4r" podStartSLOduration=3.491643809 podStartE2EDuration="22.379636471s" totalImagesPullingTime="18.887992662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:16.379142288 +0000 UTC m=+22.705703343" watchObservedRunningTime="2026-07-02 13:44:16.379636471 +0000 UTC m=+22.706197526" Jul 02 13:44:16.401692 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.399177 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9tw66" podStartSLOduration=3.561707882 podStartE2EDuration="22.399156315s" totalImagesPullingTime="18.837448433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:16.397972768 +0000 UTC m=+22.724533836" watchObservedRunningTime="2026-07-02 13:44:16.399156315 +0000 UTC m=+22.725717370" Jul 02 13:44:16.460862 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.460816 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lmznz" podStartSLOduration=2.628147144 podStartE2EDuration="21.460800017s" totalImagesPullingTime="18.832652873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:16.46027361 +0000 UTC m=+22.786834666" watchObservedRunningTime="2026-07-02 13:44:16.460800017 +0000 UTC m=+22.787361071" Jul 02 13:44:16.486366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:16.486322 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q98ws" podStartSLOduration=2.635634415 podStartE2EDuration="21.486305157s" totalImagesPullingTime="18.850670742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:16.485836918 +0000 UTC m=+22.812397975" watchObservedRunningTime="2026-07-02 13:44:16.486305157 +0000 UTC m=+22.812866225" Jul 02 13:44:17.081458 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.081240 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:17.081659 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:17.081564 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:17.354317 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.354231 2561 generic.go:356] "Generic (PLEG): container finished" podID="5d4fe52c-af75-465b-906e-3703bc2042ce" containerID="5126872d96daba1f608953355d154038c2b9ad8ebc325b4b265f711cdd34a12f" exitCode=0 Jul 02 13:44:17.355254 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.354309 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" event={"ID":"5d4fe52c-af75-465b-906e-3703bc2042ce","Type":"ContainerDied","Data":"5126872d96daba1f608953355d154038c2b9ad8ebc325b4b265f711cdd34a12f"} Jul 02 13:44:17.355698 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.355676 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rfk79" event={"ID":"c30d5842-ae96-42d5-8074-d4f321b23818","Type":"ContainerStarted","Data":"cff2db390dd367db4a7c3231b69b19c1c50917f20f5d763cae0dedc1c1e1f175"} Jul 02 13:44:17.358292 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.358275 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 13:44:17.358593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.358577 2561 generic.go:356] "Generic (PLEG): container finished" podID="bdbc7835-070d-4af2-987c-987dd8ab81fd" containerID="ac7f11e50155d8d21e3f1916bd1f150b0ff6e1e2fb7c246d708a472046555f23" exitCode=1 Jul 02 13:44:17.358680 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.358658 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" event={"ID":"bdbc7835-070d-4af2-987c-987dd8ab81fd","Type":"ContainerDied","Data":"ac7f11e50155d8d21e3f1916bd1f150b0ff6e1e2fb7c246d708a472046555f23"} Jul 02 13:44:17.358729 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.358722 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" event={"ID":"bdbc7835-070d-4af2-987c-987dd8ab81fd","Type":"ContainerStarted","Data":"9fa3d872d5a60ff2318f6760462c76a277b3f43cf7a948922bd4fcaa14499108"} Jul 02 13:44:17.358766 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.358752 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" event={"ID":"bdbc7835-070d-4af2-987c-987dd8ab81fd","Type":"ContainerStarted","Data":"06531e6a5c70cdd2da9b93242f0708e3c356930c4b53ba66c6bf3ca5234c4bc8"} Jul 02 13:44:17.358805 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.358787 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" event={"ID":"bdbc7835-070d-4af2-987c-987dd8ab81fd","Type":"ContainerStarted","Data":"256208311c64545277b133269c27ad6d0e511f3380e959c54c27566f4c5c6a70"} Jul 02 13:44:17.358849 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.358820 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" event={"ID":"bdbc7835-070d-4af2-987c-987dd8ab81fd","Type":"ContainerStarted","Data":"f8dfa5182b72647f410c79b9bb277f410ffece893c9c7aa80fcfd5a41bd0c67d"} Jul 02 13:44:17.537411 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:17.537390 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 13:44:18.084599 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:18.084572 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:18.084893 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:18.084695 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:18.084893 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:18.084738 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:18.084893 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:18.084829 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:18.143391 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:18.143206 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T13:44:17.537406332Z","UUID":"4d1d2f47-2fd6-4d08-b1a4-47bf6031a14f","Handler":null,"Name":"","Endpoint":""} Jul 02 13:44:18.144900 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:18.144875 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 13:44:18.145067 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:18.144915 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 13:44:18.362901 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:18.362829 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" event={"ID":"03a9d000-12a0-4b05-9e8d-0c89d22e59c9","Type":"ContainerStarted","Data":"7fdf095c5720343c4b519d4399266e297c9765ed5f9d19160db56c513b232a74"} Jul 02 13:44:18.468833 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:18.468798 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:44:18.469455 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:18.469427 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:44:18.488511 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:18.488467 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rfk79" podStartSLOduration=4.652052247 podStartE2EDuration="23.488453579s" totalImagesPullingTime="18.836401332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:17.392305061 +0000 UTC m=+23.718866115" watchObservedRunningTime="2026-07-02 13:44:18.488453579 +0000 UTC m=+24.815014634" Jul 02 13:44:19.080886 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:19.080698 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:19.081130 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:19.080982 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:19.367942 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:19.367853 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" event={"ID":"03a9d000-12a0-4b05-9e8d-0c89d22e59c9","Type":"ContainerStarted","Data":"58a76077c8756e1ee0661acf893a53a9e363afd56c88028e8d4424fac9a9c8a2"} Jul 02 13:44:19.371216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:19.371193 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 13:44:19.371661 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:19.371633 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" event={"ID":"bdbc7835-070d-4af2-987c-987dd8ab81fd","Type":"ContainerStarted","Data":"c43e1f1cb360b694413ac7d35eb93666c22f2b9ae7b1fe22bcedae80af0df0e7"} Jul 02 13:44:19.371775 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:19.371727 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:44:19.372347 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:19.372328 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lmznz" Jul 02 13:44:19.414446 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:19.414404 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxd44" podStartSLOduration=3.461604381 podStartE2EDuration="24.414387361s" totalImagesPullingTime="20.95278298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:19.394200915 +0000 UTC m=+25.720761972" watchObservedRunningTime="2026-07-02 13:44:19.414387361 +0000 UTC m=+25.740948415" Jul 02 13:44:20.080789 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:20.080756 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:20.081013 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:20.080794 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:20.081013 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:20.080893 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:20.081154 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:20.081052 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:21.081078 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:21.081029 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:21.081734 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:21.081167 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:22.081711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.081636 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:22.081711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.081680 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:22.082271 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:22.081753 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:22.082271 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:22.081885 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:22.379442 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.379362 2561 generic.go:356] "Generic (PLEG): container finished" podID="5d4fe52c-af75-465b-906e-3703bc2042ce" containerID="9f5fc1a60ee5e0b1d28e315e8056e456c59ae0c6351675bf310e7bf6a78ad8ea" exitCode=0 Jul 02 13:44:22.379608 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.379449 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" event={"ID":"5d4fe52c-af75-465b-906e-3703bc2042ce","Type":"ContainerDied","Data":"9f5fc1a60ee5e0b1d28e315e8056e456c59ae0c6351675bf310e7bf6a78ad8ea"} Jul 02 13:44:22.382955 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.382939 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 13:44:22.383336 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.383300 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" event={"ID":"bdbc7835-070d-4af2-987c-987dd8ab81fd","Type":"ContainerStarted","Data":"6ebd816fbd727ac6b20a429ab4e8ca2b094ae41cdde62371f6569f4ff857cff4"} Jul 02 13:44:22.383602 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.383535 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:44:22.383672 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.383605 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:44:22.383672 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.383642 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:44:22.383790 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.383762 2561 scope.go:117] "RemoveContainer" containerID="ac7f11e50155d8d21e3f1916bd1f150b0ff6e1e2fb7c246d708a472046555f23" Jul 02 13:44:22.399063 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.399029 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:44:22.399641 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:22.399624 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:44:23.081536 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:23.081469 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:23.081727 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:23.081602 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:23.388943 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:23.388870 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 13:44:23.389230 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:23.389207 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" event={"ID":"bdbc7835-070d-4af2-987c-987dd8ab81fd","Type":"ContainerStarted","Data":"dca1cec7dc43e66fd5f100f942f9cad7d29f7646a2698df43b5f0a8196bacdc1"} Jul 02 13:44:23.420818 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:23.420789 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bj8sb"] Jul 02 13:44:23.421012 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:23.420905 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:23.421012 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:23.420987 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:23.423679 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:23.423660 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p8qsr"] Jul 02 13:44:23.423793 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:23.423747 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:23.423849 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:23.423825 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:23.430057 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:23.430006 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" podStartSLOduration=10.52593136 podStartE2EDuration="29.429993609s" totalImagesPullingTime="18.904062249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:23.429949331 +0000 UTC m=+29.756510386" watchObservedRunningTime="2026-07-02 13:44:23.429993609 +0000 UTC m=+29.756554664" Jul 02 13:44:23.437019 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:23.437000 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jn2xt"] Jul 02 13:44:23.437107 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:23.437094 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:23.437188 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:23.437170 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:24.393229 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:24.393028 2561 generic.go:356] "Generic (PLEG): container finished" podID="5d4fe52c-af75-465b-906e-3703bc2042ce" containerID="e56f7c442f0203565111a38c55b6938a2e987984617192fc9e4f70ce13236190" exitCode=0 Jul 02 13:44:24.393687 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:24.393073 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" event={"ID":"5d4fe52c-af75-465b-906e-3703bc2042ce","Type":"ContainerDied","Data":"e56f7c442f0203565111a38c55b6938a2e987984617192fc9e4f70ce13236190"} Jul 02 13:44:25.081688 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:25.081652 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:25.081826 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:25.081700 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:25.081826 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:25.081700 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:25.081826 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:25.081793 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:25.081958 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:25.081875 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:25.082003 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:25.081950 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:26.409220 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:26.408941 2561 generic.go:356] "Generic (PLEG): container finished" podID="5d4fe52c-af75-465b-906e-3703bc2042ce" containerID="2d8510daabf16a7216378dc73ea88d31b53cc18f6443c9565c272ae7466a32e3" exitCode=0 Jul 02 13:44:26.410086 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:26.409022 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" event={"ID":"5d4fe52c-af75-465b-906e-3703bc2042ce","Type":"ContainerDied","Data":"2d8510daabf16a7216378dc73ea88d31b53cc18f6443c9565c272ae7466a32e3"} Jul 02 13:44:27.081330 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:27.081239 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:27.081330 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:27.081287 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:27.081330 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:27.081287 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:27.081583 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:27.081371 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p8qsr" podUID="48976aeb-7f1a-4164-a77b-9bdc07d79cb9" Jul 02 13:44:27.081583 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:27.081481 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jn2xt" podUID="4ef79fd6-3648-4b4b-bd70-38b27b45aeb5" Jul 02 13:44:27.081583 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:27.081546 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bj8sb" podUID="c0b8e79b-9ee6-4442-bb4b-5020ff3833a9" Jul 02 13:44:27.805334 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:27.805285 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:27.805810 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:27.805459 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:44:27.805810 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:27.805536 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs podName:48976aeb-7f1a-4164-a77b-9bdc07d79cb9 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:59.805515417 +0000 UTC m=+66.132076456 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs") pod "network-metrics-daemon-p8qsr" (UID: "48976aeb-7f1a-4164-a77b-9bdc07d79cb9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:44:27.906198 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:27.906162 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:27.906366 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:27.906344 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:44:27.906435 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:27.906423 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret podName:c0b8e79b-9ee6-4442-bb4b-5020ff3833a9 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:59.906401704 +0000 UTC m=+66.232962743 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret") pod "global-pull-secret-syncer-bj8sb" (UID: "c0b8e79b-9ee6-4442-bb4b-5020ff3833a9") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:44:28.007413 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:28.007379 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkx9\" (UniqueName: \"kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9\") pod \"network-check-target-jn2xt\" (UID: \"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5\") " pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:28.007591 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:28.007521 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:44:28.007591 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:28.007543 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:44:28.007591 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:28.007555 2561 projected.go:196] Error preparing data for projected volume kube-api-access-8wkx9 for pod openshift-network-diagnostics/network-check-target-jn2xt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:44:28.007757 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:28.007618 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9 podName:4ef79fd6-3648-4b4b-bd70-38b27b45aeb5 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:00.007604341 +0000 UTC m=+66.334165374 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wkx9" (UniqueName: "kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9") pod "network-check-target-jn2xt" (UID: "4ef79fd6-3648-4b4b-bd70-38b27b45aeb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:44:28.999655 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:28.999580 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-40.ec2.internal" event="NodeReady" Jul 02 13:44:29.000118 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:28.999734 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 13:44:29.041512 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.041481 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c86bf7ff8-5rw2p"] Jul 02 13:44:29.044074 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.044050 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.047758 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.047735 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 13:44:29.047910 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.047766 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 13:44:29.048319 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.048172 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 13:44:29.048319 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.048253 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j4vz9\"" Jul 02 13:44:29.054582 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.054556 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 13:44:29.054711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.054561 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2xppm"] Jul 02 13:44:29.056500 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.056483 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2xppm" Jul 02 13:44:29.059603 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.059582 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 13:44:29.059712 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.059649 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 13:44:29.060381 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.060362 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 13:44:29.060481 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.060452 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-srjlp\"" Jul 02 13:44:29.064061 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.063089 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pbgrb"] Jul 02 13:44:29.065372 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.065307 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c86bf7ff8-5rw2p"] Jul 02 13:44:29.065438 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.065405 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.067883 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.067862 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 13:44:29.067976 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.067921 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rwl8t\"" Jul 02 13:44:29.068208 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.068192 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 13:44:29.077841 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.077821 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pbgrb"] Jul 02 13:44:29.081134 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.081109 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:29.081134 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.081128 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:29.081689 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.081429 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:44:29.082809 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.082788 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2xppm"] Jul 02 13:44:29.083588 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.083570 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 13:44:29.084052 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.083803 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 13:44:29.084052 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.083808 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f4vwh\"" Jul 02 13:44:29.084052 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.083933 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bsszs\"" Jul 02 13:44:29.084052 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.083986 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 13:44:29.084324 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.084093 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 13:44:29.215096 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215033 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-image-registry-private-configuration\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.215284 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215132 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-certificates\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.215284 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215173 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-config-volume\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.215284 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqqvr\" (UniqueName: \"kubernetes.io/projected/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-kube-api-access-wqqvr\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.215284 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215273 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-bound-sa-token\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.215478 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215293 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-trusted-ca\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.215478 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215385 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.215478 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215409 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-tmp-dir\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.215478 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215439 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjxj9\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-kube-api-access-sjxj9\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.215478 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215460 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-ca-trust-extracted\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.215687 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215481 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c76f\" (UniqueName: \"kubernetes.io/projected/81787771-8105-4b8d-bec6-3ad8cca4ef30-kube-api-access-7c76f\") pod \"ingress-canary-2xppm\" (UID: \"81787771-8105-4b8d-bec6-3ad8cca4ef30\") " pod="openshift-ingress-canary/ingress-canary-2xppm" Jul 02 13:44:29.215687 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215520 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.215687 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215553 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-installation-pull-secrets\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.215687 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.215580 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81787771-8105-4b8d-bec6-3ad8cca4ef30-cert\") pod \"ingress-canary-2xppm\" (UID: \"81787771-8105-4b8d-bec6-3ad8cca4ef30\") " pod="openshift-ingress-canary/ingress-canary-2xppm" Jul 02 13:44:29.316448 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.316348 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-image-registry-private-configuration\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.316448 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.316401 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-certificates\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.316448 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.316422 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-config-volume\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.316703 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.316451 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqqvr\" (UniqueName: \"kubernetes.io/projected/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-kube-api-access-wqqvr\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.316703 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.316474 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-bound-sa-token\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.316703 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.316500 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-trusted-ca\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.316703 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.316549 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.316703 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.316574 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-tmp-dir\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.316703 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.316601 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjxj9\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-kube-api-access-sjxj9\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.316703 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.316626 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-ca-trust-extracted\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.317141 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:29.316730 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 13:44:29.317141 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:29.316806 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls podName:82c5a9c0-96fc-4f4b-8194-83a58934ba8f nodeName:}" failed. No retries permitted until 2026-07-02 13:44:29.81678712 +0000 UTC m=+36.143348169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls") pod "dns-default-pbgrb" (UID: "82c5a9c0-96fc-4f4b-8194-83a58934ba8f") : secret "dns-default-metrics-tls" not found Jul 02 13:44:29.317141 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.316975 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-tmp-dir\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.317141 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.317121 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-certificates\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.317360 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.317151 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c76f\" (UniqueName: \"kubernetes.io/projected/81787771-8105-4b8d-bec6-3ad8cca4ef30-kube-api-access-7c76f\") pod \"ingress-canary-2xppm\" (UID: \"81787771-8105-4b8d-bec6-3ad8cca4ef30\") " pod="openshift-ingress-canary/ingress-canary-2xppm" Jul 02 13:44:29.317360 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.317195 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.317360 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.317236 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-installation-pull-secrets\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.317360 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.317248 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-ca-trust-extracted\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.317360 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.317267 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81787771-8105-4b8d-bec6-3ad8cca4ef30-cert\") pod \"ingress-canary-2xppm\" (UID: \"81787771-8105-4b8d-bec6-3ad8cca4ef30\") " pod="openshift-ingress-canary/ingress-canary-2xppm" Jul 02 13:44:29.317360 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.317320 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-config-volume\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.317360 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:29.317332 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 13:44:29.317360 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:29.317347 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c86bf7ff8-5rw2p: secret "image-registry-tls" not found Jul 02 13:44:29.317737 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:29.317391 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls podName:3904b8e4-e2d9-41e1-a5c9-2021f5afb23a nodeName:}" failed. No retries permitted until 2026-07-02 13:44:29.817375374 +0000 UTC m=+36.143936410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls") pod "image-registry-7c86bf7ff8-5rw2p" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a") : secret "image-registry-tls" not found Jul 02 13:44:29.317737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.317685 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-trusted-ca\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.321108 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.321030 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-installation-pull-secrets\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.321216 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.321161 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81787771-8105-4b8d-bec6-3ad8cca4ef30-cert\") pod \"ingress-canary-2xppm\" (UID: \"81787771-8105-4b8d-bec6-3ad8cca4ef30\") " pod="openshift-ingress-canary/ingress-canary-2xppm" Jul 02 13:44:29.321335 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.321310 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-image-registry-private-configuration\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.333324 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.333290 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqqvr\" (UniqueName: \"kubernetes.io/projected/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-kube-api-access-wqqvr\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.333436 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.333390 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c76f\" (UniqueName: \"kubernetes.io/projected/81787771-8105-4b8d-bec6-3ad8cca4ef30-kube-api-access-7c76f\") pod \"ingress-canary-2xppm\" (UID: \"81787771-8105-4b8d-bec6-3ad8cca4ef30\") " pod="openshift-ingress-canary/ingress-canary-2xppm" Jul 02 13:44:29.333590 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.333572 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjxj9\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-kube-api-access-sjxj9\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.334680 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.334659 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-bound-sa-token\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.366685 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.366660 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2xppm" Jul 02 13:44:29.822359 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.822299 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:29.822565 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:29.822365 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:29.822565 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:29.822482 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 13:44:29.822565 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:29.822504 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c86bf7ff8-5rw2p: secret "image-registry-tls" not found Jul 02 13:44:29.822565 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:29.822565 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls podName:3904b8e4-e2d9-41e1-a5c9-2021f5afb23a nodeName:}" failed. No retries permitted until 2026-07-02 13:44:30.822548207 +0000 UTC m=+37.149109263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls") pod "image-registry-7c86bf7ff8-5rw2p" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a") : secret "image-registry-tls" not found Jul 02 13:44:29.822749 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:29.822483 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 13:44:29.822749 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:29.822642 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls podName:82c5a9c0-96fc-4f4b-8194-83a58934ba8f nodeName:}" failed. No retries permitted until 2026-07-02 13:44:30.822626132 +0000 UTC m=+37.149187185 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls") pod "dns-default-pbgrb" (UID: "82c5a9c0-96fc-4f4b-8194-83a58934ba8f") : secret "dns-default-metrics-tls" not found Jul 02 13:44:30.503416 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:30.503246 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2xppm"] Jul 02 13:44:30.506862 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:44:30.506820 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81787771_8105_4b8d_bec6_3ad8cca4ef30.slice/crio-cd92eccdad106c65e0909c3ffd287f89d22d4764601561adc7f51910ef8840b3 WatchSource:0}: Error finding container cd92eccdad106c65e0909c3ffd287f89d22d4764601561adc7f51910ef8840b3: Status 404 returned error can't find the container with id cd92eccdad106c65e0909c3ffd287f89d22d4764601561adc7f51910ef8840b3 Jul 02 13:44:30.831015 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:30.830919 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:30.831015 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:30.830964 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:30.831211 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:30.831071 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 13:44:30.831211 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:30.831077 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 13:44:30.831211 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:30.831126 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c86bf7ff8-5rw2p: secret "image-registry-tls" not found Jul 02 13:44:30.831211 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:30.831129 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls podName:82c5a9c0-96fc-4f4b-8194-83a58934ba8f nodeName:}" failed. No retries permitted until 2026-07-02 13:44:32.831113876 +0000 UTC m=+39.157674909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls") pod "dns-default-pbgrb" (UID: "82c5a9c0-96fc-4f4b-8194-83a58934ba8f") : secret "dns-default-metrics-tls" not found Jul 02 13:44:30.831211 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:30.831169 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls podName:3904b8e4-e2d9-41e1-a5c9-2021f5afb23a nodeName:}" failed. No retries permitted until 2026-07-02 13:44:32.831153738 +0000 UTC m=+39.157714791 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls") pod "image-registry-7c86bf7ff8-5rw2p" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a") : secret "image-registry-tls" not found Jul 02 13:44:31.425494 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:31.425458 2561 generic.go:356] "Generic (PLEG): container finished" podID="5d4fe52c-af75-465b-906e-3703bc2042ce" containerID="04b257f54ac064f36eacfdb5d1bcd3f5e272caa845fea0f2849108075995141b" exitCode=0 Jul 02 13:44:31.425685 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:31.425546 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" event={"ID":"5d4fe52c-af75-465b-906e-3703bc2042ce","Type":"ContainerDied","Data":"04b257f54ac064f36eacfdb5d1bcd3f5e272caa845fea0f2849108075995141b"} Jul 02 13:44:31.427077 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:31.427051 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2xppm" event={"ID":"81787771-8105-4b8d-bec6-3ad8cca4ef30","Type":"ContainerStarted","Data":"cd92eccdad106c65e0909c3ffd287f89d22d4764601561adc7f51910ef8840b3"} Jul 02 13:44:32.433345 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:32.433315 2561 generic.go:356] "Generic (PLEG): container finished" podID="5d4fe52c-af75-465b-906e-3703bc2042ce" containerID="a2093e7884af3efdccc2016d0ff1ccddf6e79f04ddb9eca6c11cfbc583a74f56" exitCode=0 Jul 02 13:44:32.433903 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:32.433378 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" event={"ID":"5d4fe52c-af75-465b-906e-3703bc2042ce","Type":"ContainerDied","Data":"a2093e7884af3efdccc2016d0ff1ccddf6e79f04ddb9eca6c11cfbc583a74f56"} Jul 02 13:44:32.742575 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:32.742482 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:32.845700 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:32.845666 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:32.845866 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:32.845775 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:32.845866 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:32.845834 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 13:44:32.845866 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:32.845851 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c86bf7ff8-5rw2p: secret "image-registry-tls" not found Jul 02 13:44:32.845866 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:32.845865 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 13:44:32.846033 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:32.845914 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls podName:3904b8e4-e2d9-41e1-a5c9-2021f5afb23a nodeName:}" failed. No retries permitted until 2026-07-02 13:44:36.845895897 +0000 UTC m=+43.172456937 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls") pod "image-registry-7c86bf7ff8-5rw2p" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a") : secret "image-registry-tls" not found Jul 02 13:44:32.846033 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:32.845928 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls podName:82c5a9c0-96fc-4f4b-8194-83a58934ba8f nodeName:}" failed. No retries permitted until 2026-07-02 13:44:36.845922737 +0000 UTC m=+43.172483771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls") pod "dns-default-pbgrb" (UID: "82c5a9c0-96fc-4f4b-8194-83a58934ba8f") : secret "dns-default-metrics-tls" not found Jul 02 13:44:33.437875 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:33.437781 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2xppm" event={"ID":"81787771-8105-4b8d-bec6-3ad8cca4ef30","Type":"ContainerStarted","Data":"b5c70728bc9d37aceb1f77082e606b5637b4e43972cc1865e64aeda12abd1a4a"} Jul 02 13:44:33.441219 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:33.441185 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" event={"ID":"5d4fe52c-af75-465b-906e-3703bc2042ce","Type":"ContainerStarted","Data":"1ea38fe887c7e2acdc351d8f0cad9615d7aaf974f3015b26a156a7a772a8c05e"} Jul 02 13:44:33.463476 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:33.463420 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2xppm" podStartSLOduration=1.879399743 podStartE2EDuration="4.463404444s" totalImagesPullingTime="2.584004701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:44:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:33.463029908 +0000 UTC m=+39.789590963" watchObservedRunningTime="2026-07-02 13:44:33.463404444 +0000 UTC m=+39.789965543" Jul 02 13:44:33.494487 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:33.494409 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-f8xxj" podStartSLOduration=9.442737129 podStartE2EDuration="38.494385256s" totalImagesPullingTime="29.051648127s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:33.492831923 +0000 UTC m=+39.819392978" watchObservedRunningTime="2026-07-02 13:44:33.494385256 +0000 UTC m=+39.820946313" Jul 02 13:44:34.066090 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:34.066032 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:35.858603 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:35.858564 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:35.923457 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:35.923423 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:36.875165 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:36.875124 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:36.875595 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:36.875246 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:36.875595 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:36.875285 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 13:44:36.875595 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:36.875306 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c86bf7ff8-5rw2p: secret "image-registry-tls" not found Jul 02 13:44:36.875595 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:36.875355 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 13:44:36.875595 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:36.875361 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls podName:3904b8e4-e2d9-41e1-a5c9-2021f5afb23a nodeName:}" failed. No retries permitted until 2026-07-02 13:44:44.875346443 +0000 UTC m=+51.201907475 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls") pod "image-registry-7c86bf7ff8-5rw2p" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a") : secret "image-registry-tls" not found Jul 02 13:44:36.875595 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:36.875416 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls podName:82c5a9c0-96fc-4f4b-8194-83a58934ba8f nodeName:}" failed. No retries permitted until 2026-07-02 13:44:44.875400862 +0000 UTC m=+51.201961900 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls") pod "dns-default-pbgrb" (UID: "82c5a9c0-96fc-4f4b-8194-83a58934ba8f") : secret "dns-default-metrics-tls" not found Jul 02 13:44:44.935032 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:44.934987 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:44:44.935032 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:44.935059 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:44:44.935560 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:44.935157 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 13:44:44.935560 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:44.935169 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c86bf7ff8-5rw2p: secret "image-registry-tls" not found Jul 02 13:44:44.935560 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:44.935215 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls podName:3904b8e4-e2d9-41e1-a5c9-2021f5afb23a nodeName:}" failed. No retries permitted until 2026-07-02 13:45:00.935201426 +0000 UTC m=+67.261762473 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls") pod "image-registry-7c86bf7ff8-5rw2p" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a") : secret "image-registry-tls" not found Jul 02 13:44:44.935560 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:44.935153 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 13:44:44.935560 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:44.935304 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls podName:82c5a9c0-96fc-4f4b-8194-83a58934ba8f nodeName:}" failed. No retries permitted until 2026-07-02 13:45:00.935287211 +0000 UTC m=+67.261848264 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls") pod "dns-default-pbgrb" (UID: "82c5a9c0-96fc-4f4b-8194-83a58934ba8f") : secret "dns-default-metrics-tls" not found Jul 02 13:44:45.717022 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:45.716980 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:45.817569 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:45.817533 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:45.885745 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:45.885716 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:46.736430 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:46.736400 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:46.836808 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:46.836774 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:54.404795 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:54.404757 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9gx2" Jul 02 13:44:54.575504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:54.575471 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:54.664743 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:54.664647 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:54.743666 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:54.743632 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:56.704754 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:56.704707 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:56.804290 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:56.804258 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:58.205209 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:58.205172 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:58.301406 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:58.301366 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:44:59.847877 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:59.847841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:44:59.850722 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:59.850703 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 13:44:59.858965 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:59.858946 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 13:44:59.859025 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:44:59.859015 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs podName:48976aeb-7f1a-4164-a77b-9bdc07d79cb9 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:03.858999383 +0000 UTC m=+130.185560416 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs") pod "network-metrics-daemon-p8qsr" (UID: "48976aeb-7f1a-4164-a77b-9bdc07d79cb9") : secret "metrics-daemon-secret" not found Jul 02 13:44:59.949153 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:59.949107 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:59.951545 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:59.951528 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 13:44:59.962689 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:59.962670 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c0b8e79b-9ee6-4442-bb4b-5020ff3833a9-original-pull-secret\") pod \"global-pull-secret-syncer-bj8sb\" (UID: \"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9\") " pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:44:59.993900 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:44:59.993875 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bj8sb" Jul 02 13:45:00.050461 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.050424 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkx9\" (UniqueName: \"kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9\") pod \"network-check-target-jn2xt\" (UID: \"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5\") " pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:45:00.053408 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.053384 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 13:45:00.064178 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.064153 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 13:45:00.075246 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.075221 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wkx9\" (UniqueName: \"kubernetes.io/projected/4ef79fd6-3648-4b4b-bd70-38b27b45aeb5-kube-api-access-8wkx9\") pod \"network-check-target-jn2xt\" (UID: \"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5\") " pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:45:00.122799 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.122725 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bj8sb"] Jul 02 13:45:00.125973 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:00.125941 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0b8e79b_9ee6_4442_bb4b_5020ff3833a9.slice/crio-89b90e5f6165f1ab0702ea23a2b2ed7577ef211ac7f72c323dfa04e1ae15464e WatchSource:0}: Error finding container 89b90e5f6165f1ab0702ea23a2b2ed7577ef211ac7f72c323dfa04e1ae15464e: Status 404 returned error can't find the container with id 89b90e5f6165f1ab0702ea23a2b2ed7577ef211ac7f72c323dfa04e1ae15464e Jul 02 13:45:00.307247 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.307200 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bsszs\"" Jul 02 13:45:00.315239 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.315215 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:45:00.437360 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.437325 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jn2xt"] Jul 02 13:45:00.440271 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:00.440244 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ef79fd6_3648_4b4b_bd70_38b27b45aeb5.slice/crio-1f1a56bb04b2370f8461e193faa0bcffd81a2e1a09bd52ce4374766e08247e84 WatchSource:0}: Error finding container 1f1a56bb04b2370f8461e193faa0bcffd81a2e1a09bd52ce4374766e08247e84: Status 404 returned error can't find the container with id 1f1a56bb04b2370f8461e193faa0bcffd81a2e1a09bd52ce4374766e08247e84 Jul 02 13:45:00.501771 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.501732 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jn2xt" event={"ID":"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5","Type":"ContainerStarted","Data":"1f1a56bb04b2370f8461e193faa0bcffd81a2e1a09bd52ce4374766e08247e84"} Jul 02 13:45:00.502788 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.502766 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bj8sb" event={"ID":"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9","Type":"ContainerStarted","Data":"89b90e5f6165f1ab0702ea23a2b2ed7577ef211ac7f72c323dfa04e1ae15464e"} Jul 02 13:45:00.957813 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.957765 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:45:00.957813 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:00.957820 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:45:00.958320 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:00.957931 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 13:45:00.958320 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:00.957945 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 13:45:00.958320 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:00.957957 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c86bf7ff8-5rw2p: secret "image-registry-tls" not found Jul 02 13:45:00.958320 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:00.958013 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls podName:82c5a9c0-96fc-4f4b-8194-83a58934ba8f nodeName:}" failed. No retries permitted until 2026-07-02 13:45:32.957997499 +0000 UTC m=+99.284558532 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls") pod "dns-default-pbgrb" (UID: "82c5a9c0-96fc-4f4b-8194-83a58934ba8f") : secret "dns-default-metrics-tls" not found Jul 02 13:45:00.958320 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:00.958032 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls podName:3904b8e4-e2d9-41e1-a5c9-2021f5afb23a nodeName:}" failed. No retries permitted until 2026-07-02 13:45:32.958024511 +0000 UTC m=+99.284585543 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls") pod "image-registry-7c86bf7ff8-5rw2p" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a") : secret "image-registry-tls" not found Jul 02 13:45:01.870122 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:01.870079 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:01.974089 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:01.974025 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:02.498840 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:02.498800 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:02.603995 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:02.603958 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:04.410888 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.410851 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb"] Jul 02 13:45:04.434057 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.434007 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:04.434819 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.434791 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb"] Jul 02 13:45:04.437218 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.437195 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 02 13:45:04.437777 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.437748 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 13:45:04.439317 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.439122 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4hd2s\"" Jul 02 13:45:04.439317 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.439153 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 13:45:04.439317 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.439153 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 02 13:45:04.488388 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.488354 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:04.488573 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.488440 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75pw9\" (UniqueName: \"kubernetes.io/projected/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-kube-api-access-75pw9\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:04.488573 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.488481 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:04.501953 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.501555 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk"] Jul 02 13:45:04.516784 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.516753 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" Jul 02 13:45:04.520756 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.520714 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk"] Jul 02 13:45:04.522409 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.521896 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 02 13:45:04.522409 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.521996 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 02 13:45:04.523277 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.523257 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 02 13:45:04.523385 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.523302 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-4tfzr\"" Jul 02 13:45:04.524305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.524267 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 02 13:45:04.589770 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.589737 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75pw9\" (UniqueName: \"kubernetes.io/projected/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-kube-api-access-75pw9\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:04.589960 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.589778 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:04.589960 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.589814 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6dd73fff-e40d-4e10-a678-2b6439f414db-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-pfgpk\" (UID: \"6dd73fff-e40d-4e10-a678-2b6439f414db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" Jul 02 13:45:04.589960 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.589843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6dd73fff-e40d-4e10-a678-2b6439f414db-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-pfgpk\" (UID: \"6dd73fff-e40d-4e10-a678-2b6439f414db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" Jul 02 13:45:04.589960 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.589924 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9sct\" (UniqueName: \"kubernetes.io/projected/6dd73fff-e40d-4e10-a678-2b6439f414db-kube-api-access-n9sct\") pod \"kube-storage-version-migrator-operator-79964ff6f9-pfgpk\" (UID: \"6dd73fff-e40d-4e10-a678-2b6439f414db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" Jul 02 13:45:04.589960 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:04.589939 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:04.589960 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.589951 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:04.590263 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:04.590024 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls podName:25a79a7e-2692-47b0-a51c-3a834dc4e8ff nodeName:}" failed. No retries permitted until 2026-07-02 13:45:05.090002932 +0000 UTC m=+71.416563970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-6zctb" (UID: "25a79a7e-2692-47b0-a51c-3a834dc4e8ff") : secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:04.590711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.590690 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:04.600467 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.600436 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75pw9\" (UniqueName: \"kubernetes.io/projected/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-kube-api-access-75pw9\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:04.614446 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.614405 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:04.691062 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.690959 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6dd73fff-e40d-4e10-a678-2b6439f414db-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-pfgpk\" (UID: \"6dd73fff-e40d-4e10-a678-2b6439f414db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" Jul 02 13:45:04.691062 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.691005 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6dd73fff-e40d-4e10-a678-2b6439f414db-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-pfgpk\" (UID: \"6dd73fff-e40d-4e10-a678-2b6439f414db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" Jul 02 13:45:04.691261 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.691088 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9sct\" (UniqueName: \"kubernetes.io/projected/6dd73fff-e40d-4e10-a678-2b6439f414db-kube-api-access-n9sct\") pod \"kube-storage-version-migrator-operator-79964ff6f9-pfgpk\" (UID: \"6dd73fff-e40d-4e10-a678-2b6439f414db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" Jul 02 13:45:04.691737 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.691707 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6dd73fff-e40d-4e10-a678-2b6439f414db-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-pfgpk\" (UID: \"6dd73fff-e40d-4e10-a678-2b6439f414db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" Jul 02 13:45:04.693505 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.693487 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6dd73fff-e40d-4e10-a678-2b6439f414db-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-pfgpk\" (UID: \"6dd73fff-e40d-4e10-a678-2b6439f414db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" Jul 02 13:45:04.695102 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.695072 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:04.699763 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.699740 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9sct\" (UniqueName: \"kubernetes.io/projected/6dd73fff-e40d-4e10-a678-2b6439f414db-kube-api-access-n9sct\") pod \"kube-storage-version-migrator-operator-79964ff6f9-pfgpk\" (UID: \"6dd73fff-e40d-4e10-a678-2b6439f414db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" Jul 02 13:45:04.828695 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:04.828651 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" Jul 02 13:45:05.093350 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:05.093308 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:05.093555 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:05.093479 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:05.093618 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:05.093561 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls podName:25a79a7e-2692-47b0-a51c-3a834dc4e8ff nodeName:}" failed. No retries permitted until 2026-07-02 13:45:06.093539599 +0000 UTC m=+72.420100652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-6zctb" (UID: "25a79a7e-2692-47b0-a51c-3a834dc4e8ff") : secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:05.510179 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:05.510152 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk"] Jul 02 13:45:05.513777 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:05.513749 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd73fff_e40d_4e10_a678_2b6439f414db.slice/crio-36e50147812ed7524ed231c713796f834dfbfae4eaa7416ec085e68b1e94c9a7 WatchSource:0}: Error finding container 36e50147812ed7524ed231c713796f834dfbfae4eaa7416ec085e68b1e94c9a7: Status 404 returned error can't find the container with id 36e50147812ed7524ed231c713796f834dfbfae4eaa7416ec085e68b1e94c9a7 Jul 02 13:45:05.517993 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:05.517969 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" event={"ID":"6dd73fff-e40d-4e10-a678-2b6439f414db","Type":"ContainerStarted","Data":"36e50147812ed7524ed231c713796f834dfbfae4eaa7416ec085e68b1e94c9a7"} Jul 02 13:45:05.519485 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:05.519462 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jn2xt" event={"ID":"4ef79fd6-3648-4b4b-bd70-38b27b45aeb5","Type":"ContainerStarted","Data":"fdd802e11985735b90e30928432e108bdf3a3033e41df22b363137453b3e096f"} Jul 02 13:45:05.519617 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:05.519594 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:45:05.538939 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:05.538891 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jn2xt" podStartSLOduration=65.591743772 podStartE2EDuration="1m10.538876373s" totalImagesPullingTime="4.947132601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:05.537882681 +0000 UTC m=+71.864443741" watchObservedRunningTime="2026-07-02 13:45:05.538876373 +0000 UTC m=+71.865437425" Jul 02 13:45:05.559006 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:05.558953 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bj8sb" podStartSLOduration=65.297880233 podStartE2EDuration="1m10.558933023s" totalImagesPullingTime="5.26105279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:05.558268047 +0000 UTC m=+71.884829102" watchObservedRunningTime="2026-07-02 13:45:05.558933023 +0000 UTC m=+71.885494078" Jul 02 13:45:06.101243 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:06.101210 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:06.101410 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:06.101368 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:06.101447 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:06.101428 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls podName:25a79a7e-2692-47b0-a51c-3a834dc4e8ff nodeName:}" failed. No retries permitted until 2026-07-02 13:45:08.101413384 +0000 UTC m=+74.427974416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-6zctb" (UID: "25a79a7e-2692-47b0-a51c-3a834dc4e8ff") : secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:06.525055 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:06.525006 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bj8sb" event={"ID":"c0b8e79b-9ee6-4442-bb4b-5020ff3833a9","Type":"ContainerStarted","Data":"0548cbf45e88effe67205c703379ab350adea4c6f9eb8b067966dc5165bafdfb"} Jul 02 13:45:07.530091 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:07.530033 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" event={"ID":"6dd73fff-e40d-4e10-a678-2b6439f414db","Type":"ContainerStarted","Data":"f8543bdb120705fcadcf8512ce578e0d2083da7ed3c0fc314047010fef73c682"} Jul 02 13:45:07.551033 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:07.550984 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" podStartSLOduration=1.627948468 podStartE2EDuration="3.550965754s" totalImagesPullingTime="1.923017286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:07.550444996 +0000 UTC m=+73.877006062" watchObservedRunningTime="2026-07-02 13:45:07.550965754 +0000 UTC m=+73.877526808" Jul 02 13:45:07.790645 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:07.790613 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:07.856026 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:07.855992 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:08.118229 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:08.118110 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:08.118412 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:08.118284 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:08.118412 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:08.118381 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls podName:25a79a7e-2692-47b0-a51c-3a834dc4e8ff nodeName:}" failed. No retries permitted until 2026-07-02 13:45:12.11835987 +0000 UTC m=+78.444920909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-6zctb" (UID: "25a79a7e-2692-47b0-a51c-3a834dc4e8ff") : secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:08.692669 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:08.692631 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:08.792411 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:08.792379 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:09.960112 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:09.960081 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lh2qk"] Jul 02 13:45:09.961991 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:09.961975 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:09.965077 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:09.965055 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 13:45:09.965201 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:09.965108 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k62kw\"" Jul 02 13:45:09.966027 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:09.966012 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 13:45:09.966129 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:09.966059 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 13:45:09.966129 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:09.966099 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 13:45:09.975713 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:09.975692 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lh2qk"] Jul 02 13:45:09.980779 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:09.980750 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:10.034894 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.034861 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhz5q\" (UniqueName: \"kubernetes.io/projected/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-api-access-jhz5q\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.035094 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.034908 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.035094 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.034994 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-crio-socket\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.035174 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.035099 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-data-volume\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.035174 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.035131 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.042460 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.042428 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:10.135830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.135792 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhz5q\" (UniqueName: \"kubernetes.io/projected/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-api-access-jhz5q\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.135830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.135831 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.136077 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.135866 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-crio-socket\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.136077 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.135900 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-data-volume\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.136077 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.135929 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.136077 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.135993 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-crio-socket\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.136077 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:10.136008 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 13:45:10.136077 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:10.136057 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 13:45:10.136277 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:10.136105 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm podName:d510c8d6-85b6-4dce-ae37-9cb571bf76f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:10.636084688 +0000 UTC m=+76.962645735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lh2qk" (UID: "d510c8d6-85b6-4dce-ae37-9cb571bf76f6") : configmap "kube-rbac-proxy" not found Jul 02 13:45:10.136277 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:10.136124 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls podName:d510c8d6-85b6-4dce-ae37-9cb571bf76f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:10.636115509 +0000 UTC m=+76.962676543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lh2qk" (UID: "d510c8d6-85b6-4dce-ae37-9cb571bf76f6") : secret "insights-runtime-extractor-tls" not found Jul 02 13:45:10.136366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.136288 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-data-volume\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.143295 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.143272 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-5khc9"] Jul 02 13:45:10.145270 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.145255 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-5khc9" Jul 02 13:45:10.148760 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.148742 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qxwxz\"" Jul 02 13:45:10.150168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.150128 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhz5q\" (UniqueName: \"kubernetes.io/projected/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-api-access-jhz5q\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.159451 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.159431 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-5khc9"] Jul 02 13:45:10.237265 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.237234 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl6hs\" (UniqueName: \"kubernetes.io/projected/6b5e649e-89d0-4f1c-a245-028b1900bc9b-kube-api-access-xl6hs\") pod \"network-check-source-56499946bc-5khc9\" (UID: \"6b5e649e-89d0-4f1c-a245-028b1900bc9b\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-5khc9" Jul 02 13:45:10.338365 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.338333 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl6hs\" (UniqueName: \"kubernetes.io/projected/6b5e649e-89d0-4f1c-a245-028b1900bc9b-kube-api-access-xl6hs\") pod \"network-check-source-56499946bc-5khc9\" (UID: \"6b5e649e-89d0-4f1c-a245-028b1900bc9b\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-5khc9" Jul 02 13:45:10.347303 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.347268 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl6hs\" (UniqueName: \"kubernetes.io/projected/6b5e649e-89d0-4f1c-a245-028b1900bc9b-kube-api-access-xl6hs\") pod \"network-check-source-56499946bc-5khc9\" (UID: \"6b5e649e-89d0-4f1c-a245-028b1900bc9b\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-5khc9" Jul 02 13:45:10.405844 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.405806 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:10.458033 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.458002 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-5khc9" Jul 02 13:45:10.470163 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.470127 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:10.582268 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.582239 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-5khc9"] Jul 02 13:45:10.585508 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:10.585479 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b5e649e_89d0_4f1c_a245_028b1900bc9b.slice/crio-63f2ae38806092fe1af1146f1413c999747638b6ab8496346ea4ef9b9eba0762 WatchSource:0}: Error finding container 63f2ae38806092fe1af1146f1413c999747638b6ab8496346ea4ef9b9eba0762: Status 404 returned error can't find the container with id 63f2ae38806092fe1af1146f1413c999747638b6ab8496346ea4ef9b9eba0762 Jul 02 13:45:10.640585 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.640559 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.640696 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.640632 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:10.640756 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:10.640691 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 13:45:10.640803 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:10.640766 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm podName:d510c8d6-85b6-4dce-ae37-9cb571bf76f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:11.640743182 +0000 UTC m=+77.967304236 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lh2qk" (UID: "d510c8d6-85b6-4dce-ae37-9cb571bf76f6") : configmap "kube-rbac-proxy" not found Jul 02 13:45:10.640803 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:10.640767 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 13:45:10.640906 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:10.640816 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls podName:d510c8d6-85b6-4dce-ae37-9cb571bf76f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:11.640803027 +0000 UTC m=+77.967364060 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lh2qk" (UID: "d510c8d6-85b6-4dce-ae37-9cb571bf76f6") : secret "insights-runtime-extractor-tls" not found Jul 02 13:45:10.687640 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.687600 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:10.787358 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:10.787247 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:11.542973 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:11.542937 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-5khc9" event={"ID":"6b5e649e-89d0-4f1c-a245-028b1900bc9b","Type":"ContainerStarted","Data":"5b348ccf1d524546817e0004a0650f394c4912a861b496f63e5fb273e5e339e0"} Jul 02 13:45:11.543390 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:11.542995 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-5khc9" event={"ID":"6b5e649e-89d0-4f1c-a245-028b1900bc9b","Type":"ContainerStarted","Data":"63f2ae38806092fe1af1146f1413c999747638b6ab8496346ea4ef9b9eba0762"} Jul 02 13:45:11.562115 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:11.562057 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56499946bc-5khc9" podStartSLOduration=1.562026678 podStartE2EDuration="1.562026678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:11.561147367 +0000 UTC m=+77.887708422" watchObservedRunningTime="2026-07-02 13:45:11.562026678 +0000 UTC m=+77.888587775" Jul 02 13:45:11.648788 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:11.648749 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:11.648954 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:11.648883 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 13:45:11.648954 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:11.648933 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls podName:d510c8d6-85b6-4dce-ae37-9cb571bf76f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:13.64892054 +0000 UTC m=+79.975481573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lh2qk" (UID: "d510c8d6-85b6-4dce-ae37-9cb571bf76f6") : secret "insights-runtime-extractor-tls" not found Jul 02 13:45:11.648954 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:11.648934 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:11.649108 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:11.649031 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 13:45:11.649142 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:11.649119 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm podName:d510c8d6-85b6-4dce-ae37-9cb571bf76f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:13.649104417 +0000 UTC m=+79.975665453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lh2qk" (UID: "d510c8d6-85b6-4dce-ae37-9cb571bf76f6") : configmap "kube-rbac-proxy" not found Jul 02 13:45:11.805575 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:11.805491 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:12.154642 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.154558 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:12.154785 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:12.154665 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:12.154785 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:12.154727 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls podName:25a79a7e-2692-47b0-a51c-3a834dc4e8ff nodeName:}" failed. No retries permitted until 2026-07-02 13:45:20.154713777 +0000 UTC m=+86.481274810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-6zctb" (UID: "25a79a7e-2692-47b0-a51c-3a834dc4e8ff") : secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:12.286738 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.286707 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5"] Jul 02 13:45:12.288694 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.288677 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" Jul 02 13:45:12.291526 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.291501 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 02 13:45:12.292164 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.292149 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 02 13:45:12.292241 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.292214 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 02 13:45:12.292461 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.292445 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-n495g\"" Jul 02 13:45:12.292515 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.292494 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 02 13:45:12.300288 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.300258 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5"] Jul 02 13:45:12.327960 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.327923 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:12.356926 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.356887 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9d33891a-34d1-486e-97b1-b98a8fdbfcb9-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-jhvd5\" (UID: \"9d33891a-34d1-486e-97b1-b98a8fdbfcb9\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" Jul 02 13:45:12.356926 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.356931 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg6sb\" (UniqueName: \"kubernetes.io/projected/9d33891a-34d1-486e-97b1-b98a8fdbfcb9-kube-api-access-qg6sb\") pod \"service-ca-7bb4f4bcc7-jhvd5\" (UID: \"9d33891a-34d1-486e-97b1-b98a8fdbfcb9\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" Jul 02 13:45:12.357149 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.357087 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9d33891a-34d1-486e-97b1-b98a8fdbfcb9-signing-key\") pod \"service-ca-7bb4f4bcc7-jhvd5\" (UID: \"9d33891a-34d1-486e-97b1-b98a8fdbfcb9\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" Jul 02 13:45:12.393796 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.393743 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:12.457749 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.457659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9d33891a-34d1-486e-97b1-b98a8fdbfcb9-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-jhvd5\" (UID: \"9d33891a-34d1-486e-97b1-b98a8fdbfcb9\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" Jul 02 13:45:12.457749 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.457700 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg6sb\" (UniqueName: \"kubernetes.io/projected/9d33891a-34d1-486e-97b1-b98a8fdbfcb9-kube-api-access-qg6sb\") pod \"service-ca-7bb4f4bcc7-jhvd5\" (UID: \"9d33891a-34d1-486e-97b1-b98a8fdbfcb9\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" Jul 02 13:45:12.457932 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.457889 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9d33891a-34d1-486e-97b1-b98a8fdbfcb9-signing-key\") pod \"service-ca-7bb4f4bcc7-jhvd5\" (UID: \"9d33891a-34d1-486e-97b1-b98a8fdbfcb9\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" Jul 02 13:45:12.458363 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.458342 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9d33891a-34d1-486e-97b1-b98a8fdbfcb9-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-jhvd5\" (UID: \"9d33891a-34d1-486e-97b1-b98a8fdbfcb9\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" Jul 02 13:45:12.460256 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.460235 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9d33891a-34d1-486e-97b1-b98a8fdbfcb9-signing-key\") pod \"service-ca-7bb4f4bcc7-jhvd5\" (UID: \"9d33891a-34d1-486e-97b1-b98a8fdbfcb9\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" Jul 02 13:45:12.467146 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.467123 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg6sb\" (UniqueName: \"kubernetes.io/projected/9d33891a-34d1-486e-97b1-b98a8fdbfcb9-kube-api-access-qg6sb\") pod \"service-ca-7bb4f4bcc7-jhvd5\" (UID: \"9d33891a-34d1-486e-97b1-b98a8fdbfcb9\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" Jul 02 13:45:12.531553 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.531523 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9tw66_85ec7384-3925-4a95-a743-fd26e061cb25/dns-node-resolver/0.log" Jul 02 13:45:12.597640 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.597602 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" Jul 02 13:45:12.718977 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:12.718907 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5"] Jul 02 13:45:12.721752 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:12.721727 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d33891a_34d1_486e_97b1_b98a8fdbfcb9.slice/crio-78e862385b330fbbca7853959a5e9f31b44fe76a263d8024b17f6375f8888947 WatchSource:0}: Error finding container 78e862385b330fbbca7853959a5e9f31b44fe76a263d8024b17f6375f8888947: Status 404 returned error can't find the container with id 78e862385b330fbbca7853959a5e9f31b44fe76a263d8024b17f6375f8888947 Jul 02 13:45:13.550607 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:13.550569 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" event={"ID":"9d33891a-34d1-486e-97b1-b98a8fdbfcb9","Type":"ContainerStarted","Data":"78e862385b330fbbca7853959a5e9f31b44fe76a263d8024b17f6375f8888947"} Jul 02 13:45:13.613603 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:13.613568 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:13.668828 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:13.668794 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:13.669008 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:13.668853 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:13.669008 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:13.668934 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 13:45:13.669150 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:13.669022 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 13:45:13.669150 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:13.669031 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm podName:d510c8d6-85b6-4dce-ae37-9cb571bf76f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:17.669009814 +0000 UTC m=+83.995570850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lh2qk" (UID: "d510c8d6-85b6-4dce-ae37-9cb571bf76f6") : configmap "kube-rbac-proxy" not found Jul 02 13:45:13.669150 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:13.669110 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls podName:d510c8d6-85b6-4dce-ae37-9cb571bf76f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:17.669090681 +0000 UTC m=+83.995651714 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lh2qk" (UID: "d510c8d6-85b6-4dce-ae37-9cb571bf76f6") : secret "insights-runtime-extractor-tls" not found Jul 02 13:45:13.738256 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:13.738222 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-slrnm_a4e02fa4-8728-4b3e-a301-fb8038dc5147/node-ca/0.log" Jul 02 13:45:14.551233 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:14.551199 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2xppm_81787771-8105-4b8d-bec6-3ad8cca4ef30/serve-healthcheck-canary/0.log" Jul 02 13:45:15.335191 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:15.335154 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-pfgpk_6dd73fff-e40d-4e10-a678-2b6439f414db/kube-storage-version-migrator-operator/0.log" Jul 02 13:45:15.557794 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:15.557762 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" event={"ID":"9d33891a-34d1-486e-97b1-b98a8fdbfcb9","Type":"ContainerStarted","Data":"5ff0839f95ae03ee0c50aa708bcbc9dae0324973352669f6e5cae5e33b470261"} Jul 02 13:45:15.820280 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:15.820249 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:16.005567 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:16.005529 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:16.156754 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:16.156665 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:16.336982 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:16.336869 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:16.520450 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:16.520407 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:17.550592 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:17.550561 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:17.703828 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:17.703800 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:17.703994 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:17.703841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:17.703994 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:17.703926 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 13:45:17.703994 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:17.703983 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 13:45:17.704119 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:17.703999 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm podName:d510c8d6-85b6-4dce-ae37-9cb571bf76f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:25.70398507 +0000 UTC m=+92.030546103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lh2qk" (UID: "d510c8d6-85b6-4dce-ae37-9cb571bf76f6") : configmap "kube-rbac-proxy" not found Jul 02 13:45:17.704119 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:17.704027 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls podName:d510c8d6-85b6-4dce-ae37-9cb571bf76f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:25.704014 +0000 UTC m=+92.030575034 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lh2qk" (UID: "d510c8d6-85b6-4dce-ae37-9cb571bf76f6") : secret "insights-runtime-extractor-tls" not found Jul 02 13:45:17.913601 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:17.913524 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:18.174743 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:18.174657 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:18.315825 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:18.315794 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:18.698571 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:18.698535 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:19.153840 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:19.153800 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:19.234306 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:19.234270 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:19.382443 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:19.382407 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:19.501981 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:19.501948 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:19.603331 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:19.603293 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:19.896778 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:19.896685 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:20.087468 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:20.087438 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:20.146899 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:20.146811 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:20.226173 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:20.226138 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:20.226331 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:20.226289 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:20.226369 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:20.226351 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls podName:25a79a7e-2692-47b0-a51c-3a834dc4e8ff nodeName:}" failed. No retries permitted until 2026-07-02 13:45:36.226336978 +0000 UTC m=+102.552898011 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-6zctb" (UID: "25a79a7e-2692-47b0-a51c-3a834dc4e8ff") : secret "cluster-monitoring-operator-tls" not found Jul 02 13:45:20.665347 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:20.665317 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:20.740525 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:20.740492 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:21.492424 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:21.492386 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:21.699081 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:21.699024 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:21.764213 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:21.764120 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:22.037548 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:22.037457 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:22.135622 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:22.135585 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:22.297867 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:22.297772 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:22.900873 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:22.900842 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:23.295853 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:23.295814 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:23.695627 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:23.695553 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:23.906551 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:23.906498 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:24.332570 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:24.332540 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:25.097367 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:25.097325 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:25.265259 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:25.265225 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:25.324427 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:25.324385 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:25.728478 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:25.728447 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:25.773523 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:25.773491 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:25.773677 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:25.773536 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:25.773677 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:25.773641 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 13:45:25.773797 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:25.773696 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls podName:d510c8d6-85b6-4dce-ae37-9cb571bf76f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:41.773683184 +0000 UTC m=+108.100244217 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lh2qk" (UID: "d510c8d6-85b6-4dce-ae37-9cb571bf76f6") : secret "insights-runtime-extractor-tls" not found Jul 02 13:45:25.774097 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:25.774077 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:25.901455 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:25.901425 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:26.306667 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:26.306632 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:26.498390 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:26.498360 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:27.751712 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:27.751677 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:27.910888 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:27.910857 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:28.008573 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:28.008491 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:28.109583 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:28.109549 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:28.263586 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:28.263500 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:28.468837 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:28.468805 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:28.540759 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:28.540675 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:28.623118 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:28.623084 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:29.440783 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:29.440747 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:29.646004 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:29.645969 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:30.077417 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:30.077383 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:30.177230 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:30.177198 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:33.025381 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.025350 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:45:33.025779 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.025420 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:45:33.027736 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.027711 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82c5a9c0-96fc-4f4b-8194-83a58934ba8f-metrics-tls\") pod \"dns-default-pbgrb\" (UID: \"82c5a9c0-96fc-4f4b-8194-83a58934ba8f\") " pod="openshift-dns/dns-default-pbgrb" Jul 02 13:45:33.027830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.027716 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") pod \"image-registry-7c86bf7ff8-5rw2p\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:45:33.252250 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.252171 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:33.262434 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.259621 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j4vz9\"" Jul 02 13:45:33.267074 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.267056 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:45:33.277490 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.277472 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rwl8t\"" Jul 02 13:45:33.285557 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.285533 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pbgrb" Jul 02 13:45:33.429990 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.429940 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7bb4f4bcc7-jhvd5" podStartSLOduration=19.349028519 podStartE2EDuration="21.429924203s" totalImagesPullingTime="2.080895684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:15.597217181 +0000 UTC m=+81.923778238" watchObservedRunningTime="2026-07-02 13:45:33.429924203 +0000 UTC m=+99.756485257" Jul 02 13:45:33.430770 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.430745 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c86bf7ff8-5rw2p"] Jul 02 13:45:33.434503 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:33.434467 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3904b8e4_e2d9_41e1_a5c9_2021f5afb23a.slice/crio-c32ab3b784f50c30c19f3caffc47b608e806e99644e060c1adaf9b9f25c9d573 WatchSource:0}: Error finding container c32ab3b784f50c30c19f3caffc47b608e806e99644e060c1adaf9b9f25c9d573: Status 404 returned error can't find the container with id c32ab3b784f50c30c19f3caffc47b608e806e99644e060c1adaf9b9f25c9d573 Jul 02 13:45:33.435285 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.435267 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pbgrb"] Jul 02 13:45:33.438315 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:33.438292 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c5a9c0_96fc_4f4b_8194_83a58934ba8f.slice/crio-bced0425bd4f26fa6ac420c1e45f7b7ebc74f9bca714f5c7ebbe9be1af84011f WatchSource:0}: Error finding container bced0425bd4f26fa6ac420c1e45f7b7ebc74f9bca714f5c7ebbe9be1af84011f: Status 404 returned error can't find the container with id bced0425bd4f26fa6ac420c1e45f7b7ebc74f9bca714f5c7ebbe9be1af84011f Jul 02 13:45:33.616804 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.616710 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" event={"ID":"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a","Type":"ContainerStarted","Data":"b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f"} Jul 02 13:45:33.616804 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.616771 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:45:33.616804 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.616793 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" event={"ID":"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a","Type":"ContainerStarted","Data":"c32ab3b784f50c30c19f3caffc47b608e806e99644e060c1adaf9b9f25c9d573"} Jul 02 13:45:33.617788 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.617762 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pbgrb" event={"ID":"82c5a9c0-96fc-4f4b-8194-83a58934ba8f","Type":"ContainerStarted","Data":"bced0425bd4f26fa6ac420c1e45f7b7ebc74f9bca714f5c7ebbe9be1af84011f"} Jul 02 13:45:33.638029 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:33.637980 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" podStartSLOduration=98.637963235 podStartE2EDuration="1m38.637963235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:33.637259912 +0000 UTC m=+99.963821004" watchObservedRunningTime="2026-07-02 13:45:33.637963235 +0000 UTC m=+99.964524289" Jul 02 13:45:35.626359 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:35.626322 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pbgrb" event={"ID":"82c5a9c0-96fc-4f4b-8194-83a58934ba8f","Type":"ContainerStarted","Data":"7591ae9ace542add570a19bf220fff124ab563c00fb90881025ef614771062d8"} Jul 02 13:45:35.626829 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:35.626374 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pbgrb" event={"ID":"82c5a9c0-96fc-4f4b-8194-83a58934ba8f","Type":"ContainerStarted","Data":"d415f6e8da21b4bd18a22ba07cd94f38808b70bed4a684394bf88f61688ff5dd"} Jul 02 13:45:35.626829 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:35.626458 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pbgrb" Jul 02 13:45:35.645543 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:35.645355 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pbgrb" podStartSLOduration=64.870731059 podStartE2EDuration="1m6.645337328s" totalImagesPullingTime="1.774606269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:44:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:35.645158811 +0000 UTC m=+101.971719866" watchObservedRunningTime="2026-07-02 13:45:35.645337328 +0000 UTC m=+101.971898385" Jul 02 13:45:36.061802 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:36.061774 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:36.138011 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:36.137961 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:36.253666 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:36.253631 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:36.256000 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:36.255965 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25a79a7e-2692-47b0-a51c-3a834dc4e8ff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-6zctb\" (UID: \"25a79a7e-2692-47b0-a51c-3a834dc4e8ff\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:36.528115 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:36.528083 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jn2xt" Jul 02 13:45:36.546933 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:36.546907 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" Jul 02 13:45:36.716730 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:36.716698 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb"] Jul 02 13:45:36.719756 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:36.719729 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a79a7e_2692_47b0_a51c_3a834dc4e8ff.slice/crio-bddee75150f384f4d9ae5f612fa5aeac53373580f8662205f17ab83df5816eae WatchSource:0}: Error finding container bddee75150f384f4d9ae5f612fa5aeac53373580f8662205f17ab83df5816eae: Status 404 returned error can't find the container with id bddee75150f384f4d9ae5f612fa5aeac53373580f8662205f17ab83df5816eae Jul 02 13:45:36.751408 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:36.751379 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:36.852549 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:36.852461 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:37.263888 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.263854 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf"] Jul 02 13:45:37.301109 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.301078 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf"] Jul 02 13:45:37.301290 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.301143 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:37.308639 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.308617 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 02 13:45:37.308774 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.308618 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 13:45:37.308948 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.308929 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 13:45:37.309077 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.309025 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 13:45:37.365031 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.364992 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psrqd\" (UniqueName: \"kubernetes.io/projected/09a06526-eb2a-4720-aa55-8ac879b16c47-kube-api-access-psrqd\") pod \"klusterlet-addon-workmgr-8d4c8cc47-9xqxf\" (UID: \"09a06526-eb2a-4720-aa55-8ac879b16c47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:37.365217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.365085 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09a06526-eb2a-4720-aa55-8ac879b16c47-tmp\") pod \"klusterlet-addon-workmgr-8d4c8cc47-9xqxf\" (UID: \"09a06526-eb2a-4720-aa55-8ac879b16c47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:37.365217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.365174 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/09a06526-eb2a-4720-aa55-8ac879b16c47-klusterlet-config\") pod \"klusterlet-addon-workmgr-8d4c8cc47-9xqxf\" (UID: \"09a06526-eb2a-4720-aa55-8ac879b16c47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:37.466517 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.466479 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09a06526-eb2a-4720-aa55-8ac879b16c47-tmp\") pod \"klusterlet-addon-workmgr-8d4c8cc47-9xqxf\" (UID: \"09a06526-eb2a-4720-aa55-8ac879b16c47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:37.466701 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.466533 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/09a06526-eb2a-4720-aa55-8ac879b16c47-klusterlet-config\") pod \"klusterlet-addon-workmgr-8d4c8cc47-9xqxf\" (UID: \"09a06526-eb2a-4720-aa55-8ac879b16c47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:37.466701 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.466568 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psrqd\" (UniqueName: \"kubernetes.io/projected/09a06526-eb2a-4720-aa55-8ac879b16c47-kube-api-access-psrqd\") pod \"klusterlet-addon-workmgr-8d4c8cc47-9xqxf\" (UID: \"09a06526-eb2a-4720-aa55-8ac879b16c47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:37.466965 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.466940 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09a06526-eb2a-4720-aa55-8ac879b16c47-tmp\") pod \"klusterlet-addon-workmgr-8d4c8cc47-9xqxf\" (UID: \"09a06526-eb2a-4720-aa55-8ac879b16c47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:37.469266 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.469242 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/09a06526-eb2a-4720-aa55-8ac879b16c47-klusterlet-config\") pod \"klusterlet-addon-workmgr-8d4c8cc47-9xqxf\" (UID: \"09a06526-eb2a-4720-aa55-8ac879b16c47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:37.530950 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.530871 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psrqd\" (UniqueName: \"kubernetes.io/projected/09a06526-eb2a-4720-aa55-8ac879b16c47-kube-api-access-psrqd\") pod \"klusterlet-addon-workmgr-8d4c8cc47-9xqxf\" (UID: \"09a06526-eb2a-4720-aa55-8ac879b16c47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:37.613013 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.612979 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:37.636272 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.636239 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" event={"ID":"25a79a7e-2692-47b0-a51c-3a834dc4e8ff","Type":"ContainerStarted","Data":"bddee75150f384f4d9ae5f612fa5aeac53373580f8662205f17ab83df5816eae"} Jul 02 13:45:37.709574 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.709546 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5499b6fb8f-mjsl9"] Jul 02 13:45:37.731180 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.731155 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-mjsl9" Jul 02 13:45:37.750449 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.750423 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7f8cb\"" Jul 02 13:45:37.762637 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.762610 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 13:45:37.769157 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.769132 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt8ps\" (UniqueName: \"kubernetes.io/projected/8d0f83bd-b6c3-4e4c-8051-2b44721b2d29-kube-api-access-kt8ps\") pod \"downloads-5499b6fb8f-mjsl9\" (UID: \"8d0f83bd-b6c3-4e4c-8051-2b44721b2d29\") " pod="openshift-console/downloads-5499b6fb8f-mjsl9" Jul 02 13:45:37.778267 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.778245 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 13:45:37.834066 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.833960 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:37.854523 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.854488 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-mjsl9"] Jul 02 13:45:37.870377 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.870341 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt8ps\" (UniqueName: \"kubernetes.io/projected/8d0f83bd-b6c3-4e4c-8051-2b44721b2d29-kube-api-access-kt8ps\") pod \"downloads-5499b6fb8f-mjsl9\" (UID: \"8d0f83bd-b6c3-4e4c-8051-2b44721b2d29\") " pod="openshift-console/downloads-5499b6fb8f-mjsl9" Jul 02 13:45:37.878139 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.878091 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf"] Jul 02 13:45:37.882033 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:37.882005 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a06526_eb2a_4720_aa55_8ac879b16c47.slice/crio-e40bd11d3fa4881267697721088154bf1ed9077224ee35efc16aee1de43f7d51 WatchSource:0}: Error finding container e40bd11d3fa4881267697721088154bf1ed9077224ee35efc16aee1de43f7d51: Status 404 returned error can't find the container with id e40bd11d3fa4881267697721088154bf1ed9077224ee35efc16aee1de43f7d51 Jul 02 13:45:37.921722 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:37.921691 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt8ps\" (UniqueName: \"kubernetes.io/projected/8d0f83bd-b6c3-4e4c-8051-2b44721b2d29-kube-api-access-kt8ps\") pod \"downloads-5499b6fb8f-mjsl9\" (UID: \"8d0f83bd-b6c3-4e4c-8051-2b44721b2d29\") " pod="openshift-console/downloads-5499b6fb8f-mjsl9" Jul 02 13:45:38.043666 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:38.043633 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-mjsl9" Jul 02 13:45:38.117235 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:38.117141 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:38.347286 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:38.347246 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:38.502499 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:38.502466 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:38.641314 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:38.641282 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" event={"ID":"09a06526-eb2a-4720-aa55-8ac879b16c47","Type":"ContainerStarted","Data":"e40bd11d3fa4881267697721088154bf1ed9077224ee35efc16aee1de43f7d51"} Jul 02 13:45:38.737803 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:38.737464 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-mjsl9"] Jul 02 13:45:38.740193 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:38.740168 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d0f83bd_b6c3_4e4c_8051_2b44721b2d29.slice/crio-d3790257b83900111a82945e30ef88265ce2556df2d8f24ead53ba8fd7ff5d95 WatchSource:0}: Error finding container d3790257b83900111a82945e30ef88265ce2556df2d8f24ead53ba8fd7ff5d95: Status 404 returned error can't find the container with id d3790257b83900111a82945e30ef88265ce2556df2d8f24ead53ba8fd7ff5d95 Jul 02 13:45:39.304291 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.304258 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:39.410719 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.410667 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:39.500738 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.500578 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5"] Jul 02 13:45:39.508549 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.508509 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:39.518564 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.518332 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5"] Jul 02 13:45:39.518564 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.518443 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" Jul 02 13:45:39.521604 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.521492 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 02 13:45:39.522176 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.521951 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-plf8l\"" Jul 02 13:45:39.588915 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.588784 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6422729e-0ce4-4015-8e95-d0bab0a02616-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-2n6c5\" (UID: \"6422729e-0ce4-4015-8e95-d0bab0a02616\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" Jul 02 13:45:39.657058 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.657010 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-mjsl9" event={"ID":"8d0f83bd-b6c3-4e4c-8051-2b44721b2d29","Type":"ContainerStarted","Data":"d3790257b83900111a82945e30ef88265ce2556df2d8f24ead53ba8fd7ff5d95"} Jul 02 13:45:39.659739 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.659711 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" event={"ID":"25a79a7e-2692-47b0-a51c-3a834dc4e8ff","Type":"ContainerStarted","Data":"d3a58aaf61ffdcd6b4c111b4d9741a2888a95ef33522dd0a1ecbf2dce54e98e1"} Jul 02 13:45:39.687187 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.687116 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-6zctb" podStartSLOduration=33.491125533 podStartE2EDuration="35.687095739s" totalImagesPullingTime="2.195970206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:39.685552338 +0000 UTC m=+106.012113394" watchObservedRunningTime="2026-07-02 13:45:39.687095739 +0000 UTC m=+106.013656795" Jul 02 13:45:39.690192 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:39.690161 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6422729e-0ce4-4015-8e95-d0bab0a02616-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-2n6c5\" (UID: \"6422729e-0ce4-4015-8e95-d0bab0a02616\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" Jul 02 13:45:39.690994 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:39.690500 2561 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 02 13:45:39.690994 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:39.690584 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6422729e-0ce4-4015-8e95-d0bab0a02616-tls-certificates podName:6422729e-0ce4-4015-8e95-d0bab0a02616 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:40.190564285 +0000 UTC m=+106.517125332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/6422729e-0ce4-4015-8e95-d0bab0a02616-tls-certificates") pod "prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" (UID: "6422729e-0ce4-4015-8e95-d0bab0a02616") : secret "prometheus-operator-admission-webhook-tls" not found Jul 02 13:45:40.007323 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:40.007211 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:40.099874 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:40.099791 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:40.195319 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:40.195259 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6422729e-0ce4-4015-8e95-d0bab0a02616-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-2n6c5\" (UID: \"6422729e-0ce4-4015-8e95-d0bab0a02616\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" Jul 02 13:45:40.195506 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:40.195476 2561 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 02 13:45:40.195566 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:40.195557 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6422729e-0ce4-4015-8e95-d0bab0a02616-tls-certificates podName:6422729e-0ce4-4015-8e95-d0bab0a02616 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:41.195536856 +0000 UTC m=+107.522097891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/6422729e-0ce4-4015-8e95-d0bab0a02616-tls-certificates") pod "prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" (UID: "6422729e-0ce4-4015-8e95-d0bab0a02616") : secret "prometheus-operator-admission-webhook-tls" not found Jul 02 13:45:40.284168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:40.284004 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:41.204341 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:41.204307 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6422729e-0ce4-4015-8e95-d0bab0a02616-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-2n6c5\" (UID: \"6422729e-0ce4-4015-8e95-d0bab0a02616\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" Jul 02 13:45:41.207011 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:41.206986 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6422729e-0ce4-4015-8e95-d0bab0a02616-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-2n6c5\" (UID: \"6422729e-0ce4-4015-8e95-d0bab0a02616\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" Jul 02 13:45:41.330309 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:41.330265 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" Jul 02 13:45:41.810056 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:41.809998 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:41.812761 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:41.812739 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d510c8d6-85b6-4dce-ae37-9cb571bf76f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh2qk\" (UID: \"d510c8d6-85b6-4dce-ae37-9cb571bf76f6\") " pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:42.070289 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:42.070030 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lh2qk" Jul 02 13:45:42.649859 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:42.649832 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lh2qk"] Jul 02 13:45:42.652704 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:42.652678 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd510c8d6_85b6_4dce_ae37_9cb571bf76f6.slice/crio-f0e1e0213d916a512d515eca09eb8ee7ba337c79beb091e57bb3470e8d78f19c WatchSource:0}: Error finding container f0e1e0213d916a512d515eca09eb8ee7ba337c79beb091e57bb3470e8d78f19c: Status 404 returned error can't find the container with id f0e1e0213d916a512d515eca09eb8ee7ba337c79beb091e57bb3470e8d78f19c Jul 02 13:45:42.668975 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:42.668944 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5"] Jul 02 13:45:42.672785 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:42.672751 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6422729e_0ce4_4015_8e95_d0bab0a02616.slice/crio-054c53f1bf04ef6d02346e901462357188a3e6d79b684af388deab01a337eb12 WatchSource:0}: Error finding container 054c53f1bf04ef6d02346e901462357188a3e6d79b684af388deab01a337eb12: Status 404 returned error can't find the container with id 054c53f1bf04ef6d02346e901462357188a3e6d79b684af388deab01a337eb12 Jul 02 13:45:42.675936 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:42.675879 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" event={"ID":"09a06526-eb2a-4720-aa55-8ac879b16c47","Type":"ContainerStarted","Data":"9ac3abd8a99b83adef2f83626aef368f54e72f769b48f41fe63c3a8776339492"} Jul 02 13:45:42.676102 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:42.676081 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:42.677600 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:42.677541 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" podUID="09a06526-eb2a-4720-aa55-8ac879b16c47" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.14:8000/readyz\": dial tcp 10.132.0.14:8000: connect: connection refused" Jul 02 13:45:42.678495 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:42.678463 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lh2qk" event={"ID":"d510c8d6-85b6-4dce-ae37-9cb571bf76f6","Type":"ContainerStarted","Data":"f0e1e0213d916a512d515eca09eb8ee7ba337c79beb091e57bb3470e8d78f19c"} Jul 02 13:45:42.699293 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:42.699236 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" podStartSLOduration=1.028310393 podStartE2EDuration="5.699214668s" totalImagesPullingTime="4.670904275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:42.698379404 +0000 UTC m=+109.024940460" watchObservedRunningTime="2026-07-02 13:45:42.699214668 +0000 UTC m=+109.025775722" Jul 02 13:45:43.650725 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:43.650686 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:43.683636 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:43.683607 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" event={"ID":"6422729e-0ce4-4015-8e95-d0bab0a02616","Type":"ContainerStarted","Data":"054c53f1bf04ef6d02346e901462357188a3e6d79b684af388deab01a337eb12"} Jul 02 13:45:43.684926 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:43.684905 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8d4c8cc47-9xqxf" Jul 02 13:45:43.753464 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:43.753424 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:44.111898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.111852 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54c86bdf56-jsfj7"] Jul 02 13:45:44.115087 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.114226 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.121217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.120171 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 13:45:44.121217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.120491 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 13:45:44.121217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.120710 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 13:45:44.121217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.121076 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 13:45:44.121613 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.121487 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 13:45:44.123368 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.123134 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lr5m4\"" Jul 02 13:45:44.129989 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.129966 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-service-ca\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.130107 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.130022 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-config\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.130107 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.130084 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-oauth-serving-cert\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.130234 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.130121 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85bhf\" (UniqueName: \"kubernetes.io/projected/6ffe8e9f-a3ee-4370-a09e-60312179e091-kube-api-access-85bhf\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.130234 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.130189 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-oauth-config\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.130328 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.130264 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-serving-cert\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.145160 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.144901 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54c86bdf56-jsfj7"] Jul 02 13:45:44.231645 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.231613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-service-ca\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.231827 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.231671 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-config\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.231827 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.231701 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-oauth-serving-cert\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.231827 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.231723 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85bhf\" (UniqueName: \"kubernetes.io/projected/6ffe8e9f-a3ee-4370-a09e-60312179e091-kube-api-access-85bhf\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.231827 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.231747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-oauth-config\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.231827 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.231778 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-serving-cert\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.233219 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.232723 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-config\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.233219 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.232932 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-service-ca\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.233219 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.233177 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-oauth-serving-cert\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.236149 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.235719 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-serving-cert\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.236149 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.236092 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-oauth-config\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.244308 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.244243 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85bhf\" (UniqueName: \"kubernetes.io/projected/6ffe8e9f-a3ee-4370-a09e-60312179e091-kube-api-access-85bhf\") pod \"console-54c86bdf56-jsfj7\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.428966 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.428871 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:44.587758 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.587695 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54c86bdf56-jsfj7"] Jul 02 13:45:44.592298 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:44.592259 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ffe8e9f_a3ee_4370_a09e_60312179e091.slice/crio-5073cc16aedbb7e0d869875e4caa7051d7c0793c735883971a45508ebf0c08e1 WatchSource:0}: Error finding container 5073cc16aedbb7e0d869875e4caa7051d7c0793c735883971a45508ebf0c08e1: Status 404 returned error can't find the container with id 5073cc16aedbb7e0d869875e4caa7051d7c0793c735883971a45508ebf0c08e1 Jul 02 13:45:44.691080 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.690563 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" event={"ID":"6422729e-0ce4-4015-8e95-d0bab0a02616","Type":"ContainerStarted","Data":"470f475cc5234ead688982aa292e4e7a1620685b1e67f184a4efe130dd1c0725"} Jul 02 13:45:44.691080 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.690939 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" Jul 02 13:45:44.692941 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.692908 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54c86bdf56-jsfj7" event={"ID":"6ffe8e9f-a3ee-4370-a09e-60312179e091","Type":"ContainerStarted","Data":"5073cc16aedbb7e0d869875e4caa7051d7c0793c735883971a45508ebf0c08e1"} Jul 02 13:45:44.697123 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.697101 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" Jul 02 13:45:44.710811 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:44.710755 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-2n6c5" podStartSLOduration=4.725397709 podStartE2EDuration="5.710735487s" totalImagesPullingTime="985.337778ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:44.709558206 +0000 UTC m=+111.036119261" watchObservedRunningTime="2026-07-02 13:45:44.710735487 +0000 UTC m=+111.037296543" Jul 02 13:45:45.517670 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:45.517594 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:45.589585 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:45.589484 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:45.633494 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:45.633457 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pbgrb" Jul 02 13:45:46.245860 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:46.245728 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:46.316891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:46.316781 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:48.387788 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:48.387752 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:48.869241 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:48.869123 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:48.969612 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:48.969575 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:49.720304 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:49.720266 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54c86bdf56-jsfj7" event={"ID":"6ffe8e9f-a3ee-4370-a09e-60312179e091","Type":"ContainerStarted","Data":"00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02"} Jul 02 13:45:49.721851 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:49.721808 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lh2qk" event={"ID":"d510c8d6-85b6-4dce-ae37-9cb571bf76f6","Type":"ContainerStarted","Data":"d4e1785ce595b906d40790e56718596ec260814d6163abec62f1dafea92ffb0a"} Jul 02 13:45:49.743561 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:49.743512 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54c86bdf56-jsfj7" podStartSLOduration=1.26258471 podStartE2EDuration="5.743497297s" totalImagesPullingTime="4.480912587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:49.742808791 +0000 UTC m=+116.069369847" watchObservedRunningTime="2026-07-02 13:45:49.743497297 +0000 UTC m=+116.070058348" Jul 02 13:45:49.864401 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:49.864366 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:50.035132 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.035101 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:50.664773 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.664734 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:50.729355 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.729316 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lh2qk" event={"ID":"d510c8d6-85b6-4dce-ae37-9cb571bf76f6","Type":"ContainerStarted","Data":"89fc82d39e9ecb91c2d50d2d91559fe0216b6122e26185aa07d9419bfc50756f"} Jul 02 13:45:50.937079 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.936975 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:50.947346 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.947221 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946"] Jul 02 13:45:50.956061 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.955957 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:50.959904 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.959818 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 02 13:45:50.959904 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.959834 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 02 13:45:50.960764 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.960739 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bjc8g\"" Jul 02 13:45:50.960874 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.960825 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 13:45:50.965576 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.965545 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946"] Jul 02 13:45:50.969893 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.969875 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-c5fsr"] Jul 02 13:45:50.973748 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.973719 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:50.977217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.977195 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 02 13:45:50.977758 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.977581 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cvrtj\"" Jul 02 13:45:50.977758 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.977653 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 02 13:45:50.978079 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.978059 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 02 13:45:50.988153 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.988120 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-c5fsr"] Jul 02 13:45:50.992591 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.992566 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxf7k\" (UniqueName: \"kubernetes.io/projected/28eb750e-4066-4624-9014-58b0513e0cae-kube-api-access-sxf7k\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:50.992712 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.992644 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28eb750e-4066-4624-9014-58b0513e0cae-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:50.992712 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.992685 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/28eb750e-4066-4624-9014-58b0513e0cae-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:50.992819 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:50.992717 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28eb750e-4066-4624-9014-58b0513e0cae-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:51.000910 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.000887 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qbs4t"] Jul 02 13:45:51.004868 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.004848 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.007777 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.007582 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 13:45:51.008816 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.008516 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 13:45:51.008816 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.008717 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9mx2c\"" Jul 02 13:45:51.009382 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.009205 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 13:45:51.093575 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093512 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-textfile\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.093575 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093563 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0aff0d31-ef82-40c6-9108-6c665f163422-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.093813 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28eb750e-4066-4624-9014-58b0513e0cae-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:51.093813 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093628 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0aff0d31-ef82-40c6-9108-6c665f163422-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.093813 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093652 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2hzz\" (UniqueName: \"kubernetes.io/projected/0aff0d31-ef82-40c6-9108-6c665f163422-kube-api-access-w2hzz\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.093813 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093689 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.093813 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093806 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.094107 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093844 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/28eb750e-4066-4624-9014-58b0513e0cae-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:51.094107 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093877 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-tls\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.094107 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093909 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28eb750e-4066-4624-9014-58b0513e0cae-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:51.094107 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0aff0d31-ef82-40c6-9108-6c665f163422-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.094107 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.093985 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aff0d31-ef82-40c6-9108-6c665f163422-metrics-client-ca\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.094107 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.094027 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-wtmp\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.094107 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.094082 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f58a897-d774-4145-a977-1cfee9e900f9-root\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.094457 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:51.094140 2561 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 02 13:45:51.094457 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:45:51.094238 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28eb750e-4066-4624-9014-58b0513e0cae-openshift-state-metrics-tls podName:28eb750e-4066-4624-9014-58b0513e0cae nodeName:}" failed. No retries permitted until 2026-07-02 13:45:51.594189741 +0000 UTC m=+117.920750787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/28eb750e-4066-4624-9014-58b0513e0cae-openshift-state-metrics-tls") pod "openshift-state-metrics-57f99c8b49-s5946" (UID: "28eb750e-4066-4624-9014-58b0513e0cae") : secret "openshift-state-metrics-tls" not found Jul 02 13:45:51.094457 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.094381 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28eb750e-4066-4624-9014-58b0513e0cae-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:51.094628 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.094510 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0aff0d31-ef82-40c6-9108-6c665f163422-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.094628 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.094555 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxf7k\" (UniqueName: \"kubernetes.io/projected/28eb750e-4066-4624-9014-58b0513e0cae-kube-api-access-sxf7k\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:51.094628 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.094582 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f58a897-d774-4145-a977-1cfee9e900f9-sys\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.094628 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.094609 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f58a897-d774-4145-a977-1cfee9e900f9-metrics-client-ca\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.094812 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.094631 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssx4b\" (UniqueName: \"kubernetes.io/projected/0f58a897-d774-4145-a977-1cfee9e900f9-kube-api-access-ssx4b\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.096835 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.096811 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28eb750e-4066-4624-9014-58b0513e0cae-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:51.097027 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.096988 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:51.112933 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.112895 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxf7k\" (UniqueName: \"kubernetes.io/projected/28eb750e-4066-4624-9014-58b0513e0cae-kube-api-access-sxf7k\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:51.198860 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.198772 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.198860 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.198826 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.199111 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.198890 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-tls\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.199111 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.198929 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0aff0d31-ef82-40c6-9108-6c665f163422-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.199111 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.198960 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aff0d31-ef82-40c6-9108-6c665f163422-metrics-client-ca\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.199111 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.198985 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-wtmp\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.199111 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.199028 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f58a897-d774-4145-a977-1cfee9e900f9-root\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.199111 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.199097 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0aff0d31-ef82-40c6-9108-6c665f163422-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.199424 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.199141 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f58a897-d774-4145-a977-1cfee9e900f9-sys\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.199424 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.199172 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f58a897-d774-4145-a977-1cfee9e900f9-metrics-client-ca\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.199424 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.199203 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssx4b\" (UniqueName: \"kubernetes.io/projected/0f58a897-d774-4145-a977-1cfee9e900f9-kube-api-access-ssx4b\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.199424 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.199272 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-textfile\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.199424 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.199316 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0aff0d31-ef82-40c6-9108-6c665f163422-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.199424 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.199364 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0aff0d31-ef82-40c6-9108-6c665f163422-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.199424 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.199406 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2hzz\" (UniqueName: \"kubernetes.io/projected/0aff0d31-ef82-40c6-9108-6c665f163422-kube-api-access-w2hzz\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.201857 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.201822 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.202471 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.202443 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.202804 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.202772 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f58a897-d774-4145-a977-1cfee9e900f9-sys\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.203985 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.203589 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0aff0d31-ef82-40c6-9108-6c665f163422-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.203985 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.203656 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-textfile\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.203985 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.203700 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f58a897-d774-4145-a977-1cfee9e900f9-metrics-client-ca\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.203985 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.203834 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-wtmp\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.203985 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.203974 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aff0d31-ef82-40c6-9108-6c665f163422-metrics-client-ca\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.204319 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.204244 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0aff0d31-ef82-40c6-9108-6c665f163422-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.204319 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.204305 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0f58a897-d774-4145-a977-1cfee9e900f9-root\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.205984 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.205894 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0f58a897-d774-4145-a977-1cfee9e900f9-node-exporter-tls\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.206103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.206026 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0aff0d31-ef82-40c6-9108-6c665f163422-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.207374 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.207353 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0aff0d31-ef82-40c6-9108-6c665f163422-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.209793 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.209768 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:51.215704 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.215674 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssx4b\" (UniqueName: \"kubernetes.io/projected/0f58a897-d774-4145-a977-1cfee9e900f9-kube-api-access-ssx4b\") pod \"node-exporter-qbs4t\" (UID: \"0f58a897-d774-4145-a977-1cfee9e900f9\") " pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.217373 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.217328 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2hzz\" (UniqueName: \"kubernetes.io/projected/0aff0d31-ef82-40c6-9108-6c665f163422-kube-api-access-w2hzz\") pod \"kube-state-metrics-845b954455-c5fsr\" (UID: \"0aff0d31-ef82-40c6-9108-6c665f163422\") " pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.285405 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.285368 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" Jul 02 13:45:51.316923 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.316894 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qbs4t" Jul 02 13:45:51.452378 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.452279 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:51.552616 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.552583 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:51.604099 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.604061 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/28eb750e-4066-4624-9014-58b0513e0cae-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:51.607120 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.607094 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/28eb750e-4066-4624-9014-58b0513e0cae-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-s5946\" (UID: \"28eb750e-4066-4624-9014-58b0513e0cae\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:51.711312 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.711233 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:51.870457 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.870423 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" Jul 02 13:45:51.988867 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:51.988807 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:52.082224 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.082174 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:52.088403 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.088372 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:45:52.092597 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.092560 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.095774 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.095470 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 13:45:52.095774 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.095494 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 13:45:52.095774 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.095570 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 13:45:52.095774 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.095590 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 13:45:52.095774 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.095679 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 13:45:52.095774 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.095687 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 13:45:52.096150 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.095920 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 13:45:52.096150 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.095985 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 13:45:52.096150 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.096033 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rxvbr\"" Jul 02 13:45:52.096150 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.095918 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 13:45:52.115764 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.115730 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:45:52.197862 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.197830 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:52.210051 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210007 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210205 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210067 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210205 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210147 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-web-config\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210205 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210192 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-out\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210340 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210221 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210340 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210253 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210340 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210281 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl29w\" (UniqueName: \"kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-kube-api-access-wl29w\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210340 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210309 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210515 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210362 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210515 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210399 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-volume\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210515 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210441 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210515 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210486 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.210695 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.210520 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.298506 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.298423 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:52.311434 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311391 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311600 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311439 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-web-config\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311600 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311473 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-out\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311600 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311498 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311600 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311527 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311600 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311556 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl29w\" (UniqueName: \"kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-kube-api-access-wl29w\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311600 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311584 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311913 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311611 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311913 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311637 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-volume\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311913 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311666 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311913 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311698 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311913 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311725 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.311913 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.311793 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.312898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.312772 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.313756 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.313726 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.315114 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.315088 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.316277 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.315552 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-web-config\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.317261 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.317221 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-out\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.317348 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.317268 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.317876 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.317855 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.317980 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.317878 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-volume\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.318293 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.318266 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.318814 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.318772 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.318814 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.318799 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.320249 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.320229 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.322479 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.322425 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl29w\" (UniqueName: \"kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-kube-api-access-wl29w\") pod \"alertmanager-main-0\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.406312 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.406283 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:45:52.666683 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.666535 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:52.766984 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:52.766948 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:53.271705 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:53.271671 2561 patch_prober.go:28] interesting pod/image-registry-7c86bf7ff8-5rw2p container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 13:45:53.272156 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:53.271726 2561 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" podUID="3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 13:45:53.894808 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:53.894775 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:53.996907 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:53.996866 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:54.429171 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:54.429129 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:54.429558 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:54.429193 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:54.434445 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:54.434418 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:54.443201 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:54.443174 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:54.546153 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:54.546114 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:54.625620 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:54.625592 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:45:54.748140 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:54.748110 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:45:54.781486 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:54.781432 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:54.881648 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:54.881614 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:55.073981 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.073892 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:55.485265 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.485237 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-85dd74cbb5-49dmh"] Jul 02 13:45:55.489936 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.489909 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.493652 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.493626 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 13:45:55.493652 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.493627 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 02 13:45:55.493859 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.493633 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 02 13:45:55.493859 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.493750 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pkpdj\"" Jul 02 13:45:55.493859 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.493636 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6hbtpg4tsec5e\"" Jul 02 13:45:55.494086 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.494059 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 02 13:45:55.501879 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.501858 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85dd74cbb5-49dmh"] Jul 02 13:45:55.542887 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.542854 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb03ddb5-82f7-4334-a5c4-749de4866b3e-client-ca-bundle\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.543101 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.542904 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bb03ddb5-82f7-4334-a5c4-749de4866b3e-secret-metrics-server-client-certs\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.543101 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.542938 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bb03ddb5-82f7-4334-a5c4-749de4866b3e-metrics-server-audit-profiles\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.543101 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.543030 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw9w8\" (UniqueName: \"kubernetes.io/projected/bb03ddb5-82f7-4334-a5c4-749de4866b3e-kube-api-access-dw9w8\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.543266 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.543126 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bb03ddb5-82f7-4334-a5c4-749de4866b3e-secret-metrics-server-tls\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.543266 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.543153 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb03ddb5-82f7-4334-a5c4-749de4866b3e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.543266 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.543215 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bb03ddb5-82f7-4334-a5c4-749de4866b3e-audit-log\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.551836 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.551802 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:55.644489 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.644448 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bb03ddb5-82f7-4334-a5c4-749de4866b3e-audit-log\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.644662 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.644507 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb03ddb5-82f7-4334-a5c4-749de4866b3e-client-ca-bundle\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.644724 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.644685 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bb03ddb5-82f7-4334-a5c4-749de4866b3e-secret-metrics-server-client-certs\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.644790 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.644737 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bb03ddb5-82f7-4334-a5c4-749de4866b3e-metrics-server-audit-profiles\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.644846 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.644795 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw9w8\" (UniqueName: \"kubernetes.io/projected/bb03ddb5-82f7-4334-a5c4-749de4866b3e-kube-api-access-dw9w8\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.644846 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.644839 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bb03ddb5-82f7-4334-a5c4-749de4866b3e-secret-metrics-server-tls\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.644952 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.644870 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb03ddb5-82f7-4334-a5c4-749de4866b3e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.644952 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.644928 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bb03ddb5-82f7-4334-a5c4-749de4866b3e-audit-log\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.645542 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.645514 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb03ddb5-82f7-4334-a5c4-749de4866b3e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.645912 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.645892 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bb03ddb5-82f7-4334-a5c4-749de4866b3e-metrics-server-audit-profiles\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.647621 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.647576 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bb03ddb5-82f7-4334-a5c4-749de4866b3e-secret-metrics-server-client-certs\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.647716 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.647665 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb03ddb5-82f7-4334-a5c4-749de4866b3e-client-ca-bundle\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.647716 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.647677 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bb03ddb5-82f7-4334-a5c4-749de4866b3e-secret-metrics-server-tls\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.658477 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.658452 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw9w8\" (UniqueName: \"kubernetes.io/projected/bb03ddb5-82f7-4334-a5c4-749de4866b3e-kube-api-access-dw9w8\") pod \"metrics-server-85dd74cbb5-49dmh\" (UID: \"bb03ddb5-82f7-4334-a5c4-749de4866b3e\") " pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.707182 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.707151 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:55.799998 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.799905 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:45:55.808446 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:55.808407 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:56.283553 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:56.283517 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:57.165982 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.165939 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:45:57.168643 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.168612 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:45:57.221302 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.221263 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:45:57.221485 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.221394 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.227848 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.227812 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 13:45:57.228114 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.227817 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 13:45:57.228448 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.228420 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-alg4p786vclg0\"" Jul 02 13:45:57.228554 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.228485 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 13:45:57.228646 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.228611 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 13:45:57.228726 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.228648 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 13:45:57.228780 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.228734 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 13:45:57.228839 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.228823 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 13:45:57.228974 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.228928 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vvn4n\"" Jul 02 13:45:57.229062 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.229031 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 13:45:57.229555 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.229531 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 13:45:57.229826 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.229808 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 13:45:57.230085 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.230066 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 13:45:57.230186 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.230124 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 13:45:57.230281 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.230252 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 13:45:57.231127 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.231109 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 13:45:57.361856 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.361816 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.361856 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.361864 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86ljs\" (UniqueName: \"kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-kube-api-access-86ljs\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362120 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.361893 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362120 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.361940 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362120 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.361972 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362120 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.361994 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362120 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362059 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362120 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362116 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362380 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362154 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362380 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362176 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362380 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362194 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362380 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362232 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362380 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362278 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362380 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362297 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362380 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362343 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362649 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362391 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362649 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362427 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362649 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362488 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362649 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362511 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.362649 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.362538 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463291 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463210 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463291 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463264 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463291 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463285 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463309 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463335 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463362 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463389 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463446 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463494 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463517 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86ljs\" (UniqueName: \"kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-kube-api-access-86ljs\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463546 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.463593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463596 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.464070 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463636 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.464070 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463669 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.464070 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463698 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.464070 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463728 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.464070 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463767 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.464070 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463795 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.464070 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.463827 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.464407 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.464201 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.464837 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.464805 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.464970 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.464941 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.465235 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.465210 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.466916 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.466893 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.467935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.467753 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.467935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.467853 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.467935 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.467904 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.468182 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.467937 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.468182 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.468026 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.468285 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.468206 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.468285 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.468219 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.469298 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.469263 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.469401 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.469385 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.469957 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.469884 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.470266 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.470240 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.470402 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.470378 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.471343 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.471324 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.471719 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.471696 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.475528 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.475504 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86ljs\" (UniqueName: \"kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-kube-api-access-86ljs\") pod \"prometheus-k8s-0\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.533705 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.533663 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:57.730111 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:57.730021 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f58a897_d774_4145_a977_1cfee9e900f9.slice/crio-e816ecbd23d5b910fb2daf2d8c434752fc9c7b15b26a76c108cb03b071f63e47 WatchSource:0}: Error finding container e816ecbd23d5b910fb2daf2d8c434752fc9c7b15b26a76c108cb03b071f63e47: Status 404 returned error can't find the container with id e816ecbd23d5b910fb2daf2d8c434752fc9c7b15b26a76c108cb03b071f63e47 Jul 02 13:45:57.759926 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.759842 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbs4t" event={"ID":"0f58a897-d774-4145-a977-1cfee9e900f9","Type":"ContainerStarted","Data":"e816ecbd23d5b910fb2daf2d8c434752fc9c7b15b26a76c108cb03b071f63e47"} Jul 02 13:45:57.931666 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:57.931627 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-c5fsr"] Jul 02 13:45:57.965703 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:57.965658 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aff0d31_ef82_40c6_9108_6c665f163422.slice/crio-d8fe8f0a5e22ce51a77bace16db72c63de8f6ec7d01d2d0181a626838e3c482a WatchSource:0}: Error finding container d8fe8f0a5e22ce51a77bace16db72c63de8f6ec7d01d2d0181a626838e3c482a: Status 404 returned error can't find the container with id d8fe8f0a5e22ce51a77bace16db72c63de8f6ec7d01d2d0181a626838e3c482a Jul 02 13:45:58.162517 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.161509 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:45:58.164931 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.164027 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:45:58.170127 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:58.170016 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19599919_bdf9_4801_9b53_41e23dc4fdc2.slice/crio-45c74210018fa16826123aa1b3923f8b093b603357dd3b29daec8efc66a1e84c WatchSource:0}: Error finding container 45c74210018fa16826123aa1b3923f8b093b603357dd3b29daec8efc66a1e84c: Status 404 returned error can't find the container with id 45c74210018fa16826123aa1b3923f8b093b603357dd3b29daec8efc66a1e84c Jul 02 13:45:58.170504 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:58.170280 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb98c100f_9a2d_4141_9938_2c08bfdd48a9.slice/crio-1d0699c2161f64cf101c645d89738b64c92ddd6d5f01c88f91ea6609f152a329 WatchSource:0}: Error finding container 1d0699c2161f64cf101c645d89738b64c92ddd6d5f01c88f91ea6609f152a329: Status 404 returned error can't find the container with id 1d0699c2161f64cf101c645d89738b64c92ddd6d5f01c88f91ea6609f152a329 Jul 02 13:45:58.186738 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.186665 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85dd74cbb5-49dmh"] Jul 02 13:45:58.189111 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.189089 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946"] Jul 02 13:45:58.193473 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:58.193435 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb03ddb5_82f7_4334_a5c4_749de4866b3e.slice/crio-5f9d9bce75aaca8fe7e9361cb70b15a7a010be248c980fb2eff18dd3694c9b41 WatchSource:0}: Error finding container 5f9d9bce75aaca8fe7e9361cb70b15a7a010be248c980fb2eff18dd3694c9b41: Status 404 returned error can't find the container with id 5f9d9bce75aaca8fe7e9361cb70b15a7a010be248c980fb2eff18dd3694c9b41 Jul 02 13:45:58.193884 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:45:58.193857 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28eb750e_4066_4624_9014_58b0513e0cae.slice/crio-7fe61d5ce460818dca5a4546f2b32e3be4e3da23737fe6b1c328cb639ccf0be1 WatchSource:0}: Error finding container 7fe61d5ce460818dca5a4546f2b32e3be4e3da23737fe6b1c328cb639ccf0be1: Status 404 returned error can't find the container with id 7fe61d5ce460818dca5a4546f2b32e3be4e3da23737fe6b1c328cb639ccf0be1 Jul 02 13:45:58.768157 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.766979 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-mjsl9" event={"ID":"8d0f83bd-b6c3-4e4c-8051-2b44721b2d29","Type":"ContainerStarted","Data":"45af96c0404b3e9bf8951037f9c6c7d4e1fa2ba60c1122890921b32204cde57c"} Jul 02 13:45:58.768157 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.768077 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5499b6fb8f-mjsl9" Jul 02 13:45:58.770158 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.769963 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" event={"ID":"bb03ddb5-82f7-4334-a5c4-749de4866b3e","Type":"ContainerStarted","Data":"5f9d9bce75aaca8fe7e9361cb70b15a7a010be248c980fb2eff18dd3694c9b41"} Jul 02 13:45:58.773680 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.773645 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" event={"ID":"28eb750e-4066-4624-9014-58b0513e0cae","Type":"ContainerStarted","Data":"ec9b9605877d2808f90799759e85385adaf636ee4d30303b0a8cd23ffe0d4809"} Jul 02 13:45:58.773796 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.773705 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" event={"ID":"28eb750e-4066-4624-9014-58b0513e0cae","Type":"ContainerStarted","Data":"6053d4e35f73cc97f1f7a3af755aab4193ccb4a9a75f0b5d301727dc55469bbf"} Jul 02 13:45:58.773796 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.773735 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" event={"ID":"28eb750e-4066-4624-9014-58b0513e0cae","Type":"ContainerStarted","Data":"7fe61d5ce460818dca5a4546f2b32e3be4e3da23737fe6b1c328cb639ccf0be1"} Jul 02 13:45:58.776054 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.775990 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" event={"ID":"0aff0d31-ef82-40c6-9108-6c665f163422","Type":"ContainerStarted","Data":"d8fe8f0a5e22ce51a77bace16db72c63de8f6ec7d01d2d0181a626838e3c482a"} Jul 02 13:45:58.778075 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.778033 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5499b6fb8f-mjsl9" Jul 02 13:45:58.778185 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.778099 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerStarted","Data":"45c74210018fa16826123aa1b3923f8b093b603357dd3b29daec8efc66a1e84c"} Jul 02 13:45:58.780283 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.780210 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerStarted","Data":"1d0699c2161f64cf101c645d89738b64c92ddd6d5f01c88f91ea6609f152a329"} Jul 02 13:45:58.810022 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:58.809918 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5499b6fb8f-mjsl9" podStartSLOduration=2.506628641 podStartE2EDuration="21.809894556s" totalImagesPullingTime="19.303265915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:58.787252602 +0000 UTC m=+125.113813658" watchObservedRunningTime="2026-07-02 13:45:58.809894556 +0000 UTC m=+125.136455593" Jul 02 13:45:59.795298 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:59.795216 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbs4t" event={"ID":"0f58a897-d774-4145-a977-1cfee9e900f9","Type":"ContainerDied","Data":"fb124cb965321245cf74c8a30220cab589bc01a919a6d93031c9cc5d73a5768b"} Jul 02 13:45:59.795882 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:45:59.794231 2561 generic.go:356] "Generic (PLEG): container finished" podID="0f58a897-d774-4145-a977-1cfee9e900f9" containerID="fb124cb965321245cf74c8a30220cab589bc01a919a6d93031c9cc5d73a5768b" exitCode=0 Jul 02 13:46:00.873240 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:00.873203 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c86bf7ff8-5rw2p"] Jul 02 13:46:00.969312 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:00.969281 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:01.041062 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:01.040968 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:01.128616 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:01.128487 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:03.887848 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:03.887795 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:03.943177 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:03.943144 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:46:03.945713 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:03.945679 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48976aeb-7f1a-4164-a77b-9bdc07d79cb9-metrics-certs\") pod \"network-metrics-daemon-p8qsr\" (UID: \"48976aeb-7f1a-4164-a77b-9bdc07d79cb9\") " pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:46:03.986955 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:03.986921 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:04.202407 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.202379 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f4vwh\"" Jul 02 13:46:04.209925 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.209903 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p8qsr" Jul 02 13:46:04.435351 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.435224 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p8qsr"] Jul 02 13:46:04.441366 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:46:04.441216 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48976aeb_7f1a_4164_a77b_9bdc07d79cb9.slice/crio-4396c101442799f5248891a2e710973b4eb03d8e30edafd85ce66d4013416849 WatchSource:0}: Error finding container 4396c101442799f5248891a2e710973b4eb03d8e30edafd85ce66d4013416849: Status 404 returned error can't find the container with id 4396c101442799f5248891a2e710973b4eb03d8e30edafd85ce66d4013416849 Jul 02 13:46:04.470994 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:46:04.470939 2561 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb98c100f_9a2d_4141_9938_2c08bfdd48a9.slice/crio-e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070.scope\": RecentStats: unable to find data in memory cache]" Jul 02 13:46:04.820879 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.820751 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" event={"ID":"bb03ddb5-82f7-4334-a5c4-749de4866b3e","Type":"ContainerStarted","Data":"ace91dc1c98a8c8f183f096aff05017001c166ee47ed0f2c3f2ac5c8cd3f679e"} Jul 02 13:46:04.823481 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.823447 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" event={"ID":"28eb750e-4066-4624-9014-58b0513e0cae","Type":"ContainerStarted","Data":"b27ad1e1511f98e364596cbdf299d696be79c128a698a60f2e1dfb4fd0cd0e91"} Jul 02 13:46:04.826188 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.826158 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" event={"ID":"0aff0d31-ef82-40c6-9108-6c665f163422","Type":"ContainerStarted","Data":"1bd3c3c059e8294addff0d8277b1289c31911a73414669c631eebe37d5cc0a4f"} Jul 02 13:46:04.826278 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.826232 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" event={"ID":"0aff0d31-ef82-40c6-9108-6c665f163422","Type":"ContainerStarted","Data":"558f727a1525c4fdff777eebb506ac167725c5eba1f9a84e507aa3d7fb85c58a"} Jul 02 13:46:04.826278 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.826262 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" event={"ID":"0aff0d31-ef82-40c6-9108-6c665f163422","Type":"ContainerStarted","Data":"a7f9d119c761687f382fa35b1f3262db44c95cc6b4fba41e08c618779265f692"} Jul 02 13:46:04.828551 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.828509 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lh2qk" event={"ID":"d510c8d6-85b6-4dce-ae37-9cb571bf76f6","Type":"ContainerStarted","Data":"bca8a316d988ca12c6d07c9ae15ba2cdf2ea7f81158682596e8664dff608dbb5"} Jul 02 13:46:04.831531 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.831501 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbs4t" event={"ID":"0f58a897-d774-4145-a977-1cfee9e900f9","Type":"ContainerStarted","Data":"2b9d54fcf97d8bdf24caa956248fda46be70262113295ac727abeb5b87341ee5"} Jul 02 13:46:04.831637 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.831538 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbs4t" event={"ID":"0f58a897-d774-4145-a977-1cfee9e900f9","Type":"ContainerStarted","Data":"41c09c78bc49493ea3d2db448664b651ff30ad210531ba4fd6a3dbc8d66852dc"} Jul 02 13:46:04.833314 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.833289 2561 generic.go:356] "Generic (PLEG): container finished" podID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerID="880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a" exitCode=0 Jul 02 13:46:04.833431 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.833369 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerDied","Data":"880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a"} Jul 02 13:46:04.834781 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.834754 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p8qsr" event={"ID":"48976aeb-7f1a-4164-a77b-9bdc07d79cb9","Type":"ContainerStarted","Data":"4396c101442799f5248891a2e710973b4eb03d8e30edafd85ce66d4013416849"} Jul 02 13:46:04.837540 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.837519 2561 generic.go:356] "Generic (PLEG): container finished" podID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerID="e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070" exitCode=0 Jul 02 13:46:04.837624 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.837555 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerDied","Data":"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070"} Jul 02 13:46:04.843648 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.843601 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" podStartSLOduration=3.788630361 podStartE2EDuration="9.843583435s" totalImagesPullingTime="6.054953074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:04.842529435 +0000 UTC m=+131.169090490" watchObservedRunningTime="2026-07-02 13:46:04.843583435 +0000 UTC m=+131.170144491" Jul 02 13:46:04.885579 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.885520 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-s5946" podStartSLOduration=8.997551947 podStartE2EDuration="14.885496518s" totalImagesPullingTime="5.887944571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:04.88366326 +0000 UTC m=+131.210224318" watchObservedRunningTime="2026-07-02 13:46:04.885496518 +0000 UTC m=+131.212057575" Jul 02 13:46:04.915423 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.913143 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qbs4t" podStartSLOduration=13.618657809 podStartE2EDuration="14.913090207s" totalImagesPullingTime="1.172068491s" totalInitContainerRuntime="122.363907ms" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:04.912440554 +0000 UTC m=+131.239001609" watchObservedRunningTime="2026-07-02 13:46:04.913090207 +0000 UTC m=+131.239651261" Jul 02 13:46:04.942183 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.938860 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lh2qk" podStartSLOduration=34.492795713 podStartE2EDuration="55.938836033s" totalImagesPullingTime="21.44604032s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:04.937020985 +0000 UTC m=+131.263582041" watchObservedRunningTime="2026-07-02 13:46:04.938836033 +0000 UTC m=+131.265397089" Jul 02 13:46:04.966215 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:04.966156 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-c5fsr" podStartSLOduration=8.722305714 podStartE2EDuration="14.966134495s" totalImagesPullingTime="6.243828781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:04.964317432 +0000 UTC m=+131.290878501" watchObservedRunningTime="2026-07-02 13:46:04.966134495 +0000 UTC m=+131.292695551" Jul 02 13:46:05.845967 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:05.845921 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:05.947366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:05.947323 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:06.056349 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:06.056253 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:06.163302 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:06.163266 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:06.850249 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:06.850208 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p8qsr" event={"ID":"48976aeb-7f1a-4164-a77b-9bdc07d79cb9","Type":"ContainerStarted","Data":"5a508ba8b32dd8c60adf6adfe35e994cdb2ab7dd94ddef5a06ce148b41823627"} Jul 02 13:46:06.850442 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:06.850272 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p8qsr" event={"ID":"48976aeb-7f1a-4164-a77b-9bdc07d79cb9","Type":"ContainerStarted","Data":"6cafe34dc80c9e49f4b7596ab247400e029c6238f827c9a2b1722fc853e0b899"} Jul 02 13:46:06.871201 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:06.871135 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p8qsr" podStartSLOduration=131.225434333 podStartE2EDuration="2m12.871113095s" totalImagesPullingTime="1.645678762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:43:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:06.869615044 +0000 UTC m=+133.196176104" watchObservedRunningTime="2026-07-02 13:46:06.871113095 +0000 UTC m=+133.197674152" Jul 02 13:46:06.951903 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:06.951871 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:07.053971 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:07.053898 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:07.152816 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:07.152733 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:07.226090 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:07.226028 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:07.554622 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:07.554554 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:07.650947 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:07.650912 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:07.857937 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:07.857856 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerStarted","Data":"014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c"} Jul 02 13:46:08.134694 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:08.134610 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:08.236294 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:08.236256 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:09.868596 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:09.868506 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerStarted","Data":"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df"} Jul 02 13:46:09.868596 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:09.868582 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerStarted","Data":"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3"} Jul 02 13:46:09.871531 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:09.871508 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerStarted","Data":"3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a"} Jul 02 13:46:09.871652 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:09.871561 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerStarted","Data":"a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca"} Jul 02 13:46:09.871652 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:09.871589 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerStarted","Data":"c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf"} Jul 02 13:46:09.871652 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:09.871613 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerStarted","Data":"9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f"} Jul 02 13:46:11.885995 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:11.885960 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerStarted","Data":"f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682"} Jul 02 13:46:11.918105 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:11.918049 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=9.748610946 podStartE2EDuration="19.918018145s" totalImagesPullingTime="10.118995029s" totalInitContainerRuntime="50.41217ms" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:11.916282354 +0000 UTC m=+138.242843408" watchObservedRunningTime="2026-07-02 13:46:11.918018145 +0000 UTC m=+138.244579200" Jul 02 13:46:12.893638 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:12.893549 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerStarted","Data":"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e"} Jul 02 13:46:12.893638 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:12.893614 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerStarted","Data":"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d"} Jul 02 13:46:12.894175 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:12.893643 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerStarted","Data":"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46"} Jul 02 13:46:12.894175 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:12.893670 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerStarted","Data":"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec"} Jul 02 13:46:12.938804 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:12.938752 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.57869109 podStartE2EDuration="15.938726803s" totalImagesPullingTime="13.316035987s" totalInitContainerRuntime="43.999726ms" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:12.937120438 +0000 UTC m=+139.263681492" watchObservedRunningTime="2026-07-02 13:46:12.938726803 +0000 UTC m=+139.265287862" Jul 02 13:46:12.958551 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:12.958525 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54c86bdf56-jsfj7"] Jul 02 13:46:13.125394 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:13.125361 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:13.228224 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:13.228139 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:13.325658 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:13.325624 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:14.937320 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:14.937275 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:15.010498 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:15.010457 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:15.800242 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:15.800208 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:46:15.800447 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:15.800281 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:46:17.534799 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:17.534757 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:46:18.921684 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:18.921653 2561 generic.go:356] "Generic (PLEG): container finished" podID="6dd73fff-e40d-4e10-a678-2b6439f414db" containerID="f8543bdb120705fcadcf8512ce578e0d2083da7ed3c0fc314047010fef73c682" exitCode=0 Jul 02 13:46:18.922186 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:18.921733 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" event={"ID":"6dd73fff-e40d-4e10-a678-2b6439f414db","Type":"ContainerDied","Data":"f8543bdb120705fcadcf8512ce578e0d2083da7ed3c0fc314047010fef73c682"} Jul 02 13:46:18.922263 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:18.922191 2561 scope.go:117] "RemoveContainer" containerID="f8543bdb120705fcadcf8512ce578e0d2083da7ed3c0fc314047010fef73c682" Jul 02 13:46:19.927218 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:19.927185 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-pfgpk" event={"ID":"6dd73fff-e40d-4e10-a678-2b6439f414db","Type":"ContainerStarted","Data":"f41f2bd7284505e709f25a5b7bd8523e26cc27f07b612c3c279020bdcdbacb11"} Jul 02 13:46:20.882398 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:20.882354 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:20.947294 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:20.947265 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:25.899823 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:25.899760 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" podUID="3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" containerName="registry" containerID="cri-o://b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f" gracePeriod=30 Jul 02 13:46:26.184384 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.184359 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:46:26.248550 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.247422 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-image-registry-private-configuration\") pod \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " Jul 02 13:46:26.248550 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.247466 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-certificates\") pod \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " Jul 02 13:46:26.248550 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.247531 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-ca-trust-extracted\") pod \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " Jul 02 13:46:26.248550 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.247563 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjxj9\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-kube-api-access-sjxj9\") pod \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " Jul 02 13:46:26.248550 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.247592 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-bound-sa-token\") pod \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " Jul 02 13:46:26.248550 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.247625 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") pod \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " Jul 02 13:46:26.248550 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.247649 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-installation-pull-secrets\") pod \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " Jul 02 13:46:26.248550 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.247701 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-trusted-ca\") pod \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\" (UID: \"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a\") " Jul 02 13:46:26.248550 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.248260 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:46:26.249137 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.248981 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:46:26.249348 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.249302 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-trusted-ca\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:26.249348 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.249328 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-certificates\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:26.250503 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.250472 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:46:26.252208 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.252183 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-kube-api-access-sjxj9" (OuterVolumeSpecName: "kube-api-access-sjxj9") pod "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a"). InnerVolumeSpecName "kube-api-access-sjxj9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:46:26.252445 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.252412 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:46:26.252665 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.252631 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:46:26.252916 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.252877 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:46:26.260202 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.260173 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" (UID: "3904b8e4-e2d9-41e1-a5c9-2021f5afb23a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:46:26.350401 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.350369 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjxj9\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-kube-api-access-sjxj9\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:26.350401 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.350396 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-bound-sa-token\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:26.350401 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.350405 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-registry-tls\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:26.350614 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.350414 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-installation-pull-secrets\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:26.350614 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.350424 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-image-registry-private-configuration\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:26.350614 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.350434 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a-ca-trust-extracted\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:26.956377 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.956345 2561 generic.go:356] "Generic (PLEG): container finished" podID="3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" containerID="b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f" exitCode=0 Jul 02 13:46:26.956965 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.956392 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" event={"ID":"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a","Type":"ContainerDied","Data":"b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f"} Jul 02 13:46:26.956965 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.956430 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" event={"ID":"3904b8e4-e2d9-41e1-a5c9-2021f5afb23a","Type":"ContainerDied","Data":"c32ab3b784f50c30c19f3caffc47b608e806e99644e060c1adaf9b9f25c9d573"} Jul 02 13:46:26.956965 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.956449 2561 scope.go:117] "RemoveContainer" containerID="b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f" Jul 02 13:46:26.956965 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.956395 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c86bf7ff8-5rw2p" Jul 02 13:46:26.965001 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.964983 2561 scope.go:117] "RemoveContainer" containerID="b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f" Jul 02 13:46:26.965278 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:46:26.965257 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f\": container with ID starting with b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f not found: ID does not exist" containerID="b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f" Jul 02 13:46:26.965361 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.965288 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f"} err="failed to get container status \"b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f\": rpc error: code = NotFound desc = could not find container \"b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f\": container with ID starting with b301f7abcfa6e43c92fdf162ba6ff6e0023b5955bc638f7f59bf370ba558a34f not found: ID does not exist" Jul 02 13:46:26.977959 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.977927 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c86bf7ff8-5rw2p"] Jul 02 13:46:26.979891 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:26.979869 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7c86bf7ff8-5rw2p"] Jul 02 13:46:28.085683 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:28.085641 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" path="/var/lib/kubelet/pods/3904b8e4-e2d9-41e1-a5c9-2021f5afb23a/volumes" Jul 02 13:46:35.805350 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:35.805311 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:46:35.809276 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:35.809256 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-85dd74cbb5-49dmh" Jul 02 13:46:35.881005 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:35.880962 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:35.943936 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:35.943900 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:46:37.979978 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:37.979777 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-54c86bdf56-jsfj7" podUID="6ffe8e9f-a3ee-4370-a09e-60312179e091" containerName="console" containerID="cri-o://00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02" gracePeriod=15 Jul 02 13:46:38.297051 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.297011 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:46:38.358993 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.358229 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-oauth-config\") pod \"6ffe8e9f-a3ee-4370-a09e-60312179e091\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " Jul 02 13:46:38.358993 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.358310 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-service-ca\") pod \"6ffe8e9f-a3ee-4370-a09e-60312179e091\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " Jul 02 13:46:38.358993 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.358359 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85bhf\" (UniqueName: \"kubernetes.io/projected/6ffe8e9f-a3ee-4370-a09e-60312179e091-kube-api-access-85bhf\") pod \"6ffe8e9f-a3ee-4370-a09e-60312179e091\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " Jul 02 13:46:38.358993 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.358393 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-config\") pod \"6ffe8e9f-a3ee-4370-a09e-60312179e091\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " Jul 02 13:46:38.358993 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.358454 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-serving-cert\") pod \"6ffe8e9f-a3ee-4370-a09e-60312179e091\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " Jul 02 13:46:38.358993 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.358514 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-oauth-serving-cert\") pod \"6ffe8e9f-a3ee-4370-a09e-60312179e091\" (UID: \"6ffe8e9f-a3ee-4370-a09e-60312179e091\") " Jul 02 13:46:38.359470 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.359396 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-service-ca" (OuterVolumeSpecName: "service-ca") pod "6ffe8e9f-a3ee-4370-a09e-60312179e091" (UID: "6ffe8e9f-a3ee-4370-a09e-60312179e091"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:46:38.359793 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.359759 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-config" (OuterVolumeSpecName: "console-config") pod "6ffe8e9f-a3ee-4370-a09e-60312179e091" (UID: "6ffe8e9f-a3ee-4370-a09e-60312179e091"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:46:38.359897 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.359790 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6ffe8e9f-a3ee-4370-a09e-60312179e091" (UID: "6ffe8e9f-a3ee-4370-a09e-60312179e091"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:46:38.363386 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.363358 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ffe8e9f-a3ee-4370-a09e-60312179e091-kube-api-access-85bhf" (OuterVolumeSpecName: "kube-api-access-85bhf") pod "6ffe8e9f-a3ee-4370-a09e-60312179e091" (UID: "6ffe8e9f-a3ee-4370-a09e-60312179e091"). InnerVolumeSpecName "kube-api-access-85bhf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:46:38.363921 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.363891 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6ffe8e9f-a3ee-4370-a09e-60312179e091" (UID: "6ffe8e9f-a3ee-4370-a09e-60312179e091"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:46:38.367243 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.366130 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6ffe8e9f-a3ee-4370-a09e-60312179e091" (UID: "6ffe8e9f-a3ee-4370-a09e-60312179e091"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:46:38.459957 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.459921 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-oauth-serving-cert\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:38.459957 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.459955 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-oauth-config\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:38.459957 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.459964 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-service-ca\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:38.460204 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.459974 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85bhf\" (UniqueName: \"kubernetes.io/projected/6ffe8e9f-a3ee-4370-a09e-60312179e091-kube-api-access-85bhf\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:38.460204 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.459983 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-config\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:38.460204 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:38.459992 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ffe8e9f-a3ee-4370-a09e-60312179e091-console-serving-cert\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:46:39.013060 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:39.009141 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ffe8e9f-a3ee-4370-a09e-60312179e091" containerID="00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02" exitCode=0 Jul 02 13:46:39.013060 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:39.009260 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54c86bdf56-jsfj7" event={"ID":"6ffe8e9f-a3ee-4370-a09e-60312179e091","Type":"ContainerDied","Data":"00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02"} Jul 02 13:46:39.013060 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:39.009348 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54c86bdf56-jsfj7" event={"ID":"6ffe8e9f-a3ee-4370-a09e-60312179e091","Type":"ContainerDied","Data":"5073cc16aedbb7e0d869875e4caa7051d7c0793c735883971a45508ebf0c08e1"} Jul 02 13:46:39.013060 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:39.009378 2561 scope.go:117] "RemoveContainer" containerID="00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02" Jul 02 13:46:39.013060 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:39.009555 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54c86bdf56-jsfj7" Jul 02 13:46:39.030794 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:39.030776 2561 scope.go:117] "RemoveContainer" containerID="00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02" Jul 02 13:46:39.031033 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:46:39.031014 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02\": container with ID starting with 00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02 not found: ID does not exist" containerID="00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02" Jul 02 13:46:39.031115 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:39.031057 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02"} err="failed to get container status \"00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02\": rpc error: code = NotFound desc = could not find container \"00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02\": container with ID starting with 00a1016321e8a68b06af723f4401dc7387d595105cf84354ec42000f97362c02 not found: ID does not exist" Jul 02 13:46:39.040873 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:39.040849 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54c86bdf56-jsfj7"] Jul 02 13:46:39.048482 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:39.048455 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54c86bdf56-jsfj7"] Jul 02 13:46:40.086654 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:40.086623 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ffe8e9f-a3ee-4370-a09e-60312179e091" path="/var/lib/kubelet/pods/6ffe8e9f-a3ee-4370-a09e-60312179e091/volumes" Jul 02 13:46:57.534293 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:57.534250 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:46:57.553286 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:57.553262 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:46:58.104489 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:46:58.104464 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:07.091625 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:07.091585 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:07.193346 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:07.193311 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:09.812298 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:09.812262 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:09.913675 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:09.913622 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:10.013421 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:10.013388 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:10.572168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:10.572129 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:10.670281 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:10.670249 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:11.379801 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:11.379767 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:47:11.380225 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:11.380083 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="alertmanager" containerID="cri-o://014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c" gracePeriod=120 Jul 02 13:47:11.380225 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:11.380126 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy-web" containerID="cri-o://c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf" gracePeriod=120 Jul 02 13:47:11.380225 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:11.380165 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="config-reloader" containerID="cri-o://9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f" gracePeriod=120 Jul 02 13:47:11.380392 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:11.380203 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="prom-label-proxy" containerID="cri-o://f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682" gracePeriod=120 Jul 02 13:47:11.380392 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:11.380127 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy-metric" containerID="cri-o://3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a" gracePeriod=120 Jul 02 13:47:11.380392 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:11.380160 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy" containerID="cri-o://a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca" gracePeriod=120 Jul 02 13:47:12.144179 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.144147 2561 generic.go:356] "Generic (PLEG): container finished" podID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerID="f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682" exitCode=0 Jul 02 13:47:12.144179 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.144172 2561 generic.go:356] "Generic (PLEG): container finished" podID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerID="a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca" exitCode=0 Jul 02 13:47:12.144179 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.144179 2561 generic.go:356] "Generic (PLEG): container finished" podID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerID="9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f" exitCode=0 Jul 02 13:47:12.144179 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.144184 2561 generic.go:356] "Generic (PLEG): container finished" podID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerID="014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c" exitCode=0 Jul 02 13:47:12.144499 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.144229 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerDied","Data":"f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682"} Jul 02 13:47:12.144499 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.144309 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerDied","Data":"a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca"} Jul 02 13:47:12.144499 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.144347 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerDied","Data":"9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f"} Jul 02 13:47:12.144499 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.144385 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerDied","Data":"014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c"} Jul 02 13:47:12.627978 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.627949 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:12.659491 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.659462 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl29w\" (UniqueName: \"kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-kube-api-access-wl29w\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.659677 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.659507 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-main-tls\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.659677 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.659526 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-web-config\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.659677 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.659550 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.659677 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.659628 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-cluster-tls-config\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.659901 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.659686 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.659901 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.659735 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-metrics-client-ca\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.660006 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.659985 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-main-db\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.660102 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.660023 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-web\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.660102 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.660081 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-volume\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.660205 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.660163 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-out\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.660205 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.660194 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-trusted-ca-bundle\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.660299 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.660226 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-tls-assets\") pod \"19599919-bdf9-4801-9b53-41e23dc4fdc2\" (UID: \"19599919-bdf9-4801-9b53-41e23dc4fdc2\") " Jul 02 13:47:12.660654 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.660594 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:47:12.660918 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.660865 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:47:12.660999 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.660924 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:47:12.664013 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.663756 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:12.664013 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.663857 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-kube-api-access-wl29w" (OuterVolumeSpecName: "kube-api-access-wl29w") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "kube-api-access-wl29w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:47:12.664013 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.663931 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:12.664013 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.663951 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-volume" (OuterVolumeSpecName: "config-volume") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:12.664348 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.664115 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:12.664348 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.664252 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:47:12.665123 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.664695 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:12.671757 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.671730 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-out" (OuterVolumeSpecName: "config-out") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:47:12.673601 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.673578 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:12.680163 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.680137 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-web-config" (OuterVolumeSpecName: "web-config") pod "19599919-bdf9-4801-9b53-41e23dc4fdc2" (UID: "19599919-bdf9-4801-9b53-41e23dc4fdc2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:12.760859 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.760815 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-out\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.760859 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.760853 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.760859 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.760866 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-tls-assets\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.761168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.760879 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl29w\" (UniqueName: \"kubernetes.io/projected/19599919-bdf9-4801-9b53-41e23dc4fdc2-kube-api-access-wl29w\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.761168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.760892 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-main-tls\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.761168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.760932 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-web-config\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.761168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.760945 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.761168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.760958 2561 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-cluster-tls-config\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.761168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.760970 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.761168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.760984 2561 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19599919-bdf9-4801-9b53-41e23dc4fdc2-metrics-client-ca\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.761168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.760996 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19599919-bdf9-4801-9b53-41e23dc4fdc2-alertmanager-main-db\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.761168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.761008 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:12.761168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:12.761021 2561 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19599919-bdf9-4801-9b53-41e23dc4fdc2-config-volume\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:13.151448 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.151346 2561 generic.go:356] "Generic (PLEG): container finished" podID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerID="3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a" exitCode=0 Jul 02 13:47:13.151448 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.151379 2561 generic.go:356] "Generic (PLEG): container finished" podID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerID="c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf" exitCode=0 Jul 02 13:47:13.151448 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.151426 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerDied","Data":"3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a"} Jul 02 13:47:13.151721 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.151458 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.151721 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.151515 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerDied","Data":"c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf"} Jul 02 13:47:13.151721 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.151556 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19599919-bdf9-4801-9b53-41e23dc4fdc2","Type":"ContainerDied","Data":"45c74210018fa16826123aa1b3923f8b093b603357dd3b29daec8efc66a1e84c"} Jul 02 13:47:13.151721 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.151586 2561 scope.go:117] "RemoveContainer" containerID="f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682" Jul 02 13:47:13.160333 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.160314 2561 scope.go:117] "RemoveContainer" containerID="3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a" Jul 02 13:47:13.169164 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.169119 2561 scope.go:117] "RemoveContainer" containerID="a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca" Jul 02 13:47:13.175955 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.175938 2561 scope.go:117] "RemoveContainer" containerID="c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf" Jul 02 13:47:13.182804 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.182781 2561 scope.go:117] "RemoveContainer" containerID="9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f" Jul 02 13:47:13.185215 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.185192 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:47:13.189456 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.189437 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:47:13.190866 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.190853 2561 scope.go:117] "RemoveContainer" containerID="014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c" Jul 02 13:47:13.197711 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.197695 2561 scope.go:117] "RemoveContainer" containerID="880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a" Jul 02 13:47:13.204291 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.204274 2561 scope.go:117] "RemoveContainer" containerID="f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682" Jul 02 13:47:13.204545 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:13.204524 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682\": container with ID starting with f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682 not found: ID does not exist" containerID="f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682" Jul 02 13:47:13.204628 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.204552 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682"} err="failed to get container status \"f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682\": rpc error: code = NotFound desc = could not find container \"f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682\": container with ID starting with f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682 not found: ID does not exist" Jul 02 13:47:13.204628 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.204571 2561 scope.go:117] "RemoveContainer" containerID="3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a" Jul 02 13:47:13.204838 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:13.204816 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a\": container with ID starting with 3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a not found: ID does not exist" containerID="3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a" Jul 02 13:47:13.204912 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.204850 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a"} err="failed to get container status \"3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a\": rpc error: code = NotFound desc = could not find container \"3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a\": container with ID starting with 3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a not found: ID does not exist" Jul 02 13:47:13.204912 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.204873 2561 scope.go:117] "RemoveContainer" containerID="a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca" Jul 02 13:47:13.205163 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:13.205141 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca\": container with ID starting with a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca not found: ID does not exist" containerID="a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca" Jul 02 13:47:13.205219 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.205171 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca"} err="failed to get container status \"a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca\": rpc error: code = NotFound desc = could not find container \"a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca\": container with ID starting with a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca not found: ID does not exist" Jul 02 13:47:13.205219 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.205189 2561 scope.go:117] "RemoveContainer" containerID="c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf" Jul 02 13:47:13.205420 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:13.205405 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf\": container with ID starting with c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf not found: ID does not exist" containerID="c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf" Jul 02 13:47:13.205459 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.205434 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf"} err="failed to get container status \"c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf\": rpc error: code = NotFound desc = could not find container \"c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf\": container with ID starting with c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf not found: ID does not exist" Jul 02 13:47:13.205459 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.205450 2561 scope.go:117] "RemoveContainer" containerID="9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f" Jul 02 13:47:13.205664 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:13.205649 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f\": container with ID starting with 9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f not found: ID does not exist" containerID="9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f" Jul 02 13:47:13.205698 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.205668 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f"} err="failed to get container status \"9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f\": rpc error: code = NotFound desc = could not find container \"9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f\": container with ID starting with 9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f not found: ID does not exist" Jul 02 13:47:13.205698 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.205681 2561 scope.go:117] "RemoveContainer" containerID="014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c" Jul 02 13:47:13.205858 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:13.205844 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c\": container with ID starting with 014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c not found: ID does not exist" containerID="014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c" Jul 02 13:47:13.205897 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.205863 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c"} err="failed to get container status \"014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c\": rpc error: code = NotFound desc = could not find container \"014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c\": container with ID starting with 014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c not found: ID does not exist" Jul 02 13:47:13.205897 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.205874 2561 scope.go:117] "RemoveContainer" containerID="880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a" Jul 02 13:47:13.206100 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:13.206085 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a\": container with ID starting with 880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a not found: ID does not exist" containerID="880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a" Jul 02 13:47:13.206144 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.206104 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a"} err="failed to get container status \"880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a\": rpc error: code = NotFound desc = could not find container \"880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a\": container with ID starting with 880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a not found: ID does not exist" Jul 02 13:47:13.206144 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.206116 2561 scope.go:117] "RemoveContainer" containerID="f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682" Jul 02 13:47:13.206317 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.206302 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682"} err="failed to get container status \"f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682\": rpc error: code = NotFound desc = could not find container \"f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682\": container with ID starting with f9206471be2ad331d425da8dbae3e1f7a10550cd081b90374cf6100a2c880682 not found: ID does not exist" Jul 02 13:47:13.206358 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.206317 2561 scope.go:117] "RemoveContainer" containerID="3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a" Jul 02 13:47:13.206499 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.206484 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a"} err="failed to get container status \"3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a\": rpc error: code = NotFound desc = could not find container \"3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a\": container with ID starting with 3102bacfd71b7f57fc8dfb51916c5f6a259deee4f1e503a341b8dfa1306f1c0a not found: ID does not exist" Jul 02 13:47:13.206542 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.206500 2561 scope.go:117] "RemoveContainer" containerID="a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca" Jul 02 13:47:13.206676 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.206661 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca"} err="failed to get container status \"a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca\": rpc error: code = NotFound desc = could not find container \"a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca\": container with ID starting with a312cb96f433d8289ff2869ed5bcf7b61592a101d1402365efb6d859bde830ca not found: ID does not exist" Jul 02 13:47:13.206717 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.206676 2561 scope.go:117] "RemoveContainer" containerID="c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf" Jul 02 13:47:13.206855 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.206840 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf"} err="failed to get container status \"c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf\": rpc error: code = NotFound desc = could not find container \"c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf\": container with ID starting with c25a337aa434c4ff40df0dfdda36e772c456623865012530146f82c644723ccf not found: ID does not exist" Jul 02 13:47:13.206898 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.206855 2561 scope.go:117] "RemoveContainer" containerID="9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f" Jul 02 13:47:13.207029 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.207013 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f"} err="failed to get container status \"9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f\": rpc error: code = NotFound desc = could not find container \"9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f\": container with ID starting with 9d74e0631be9fcd2f0ac4395ea0605097f7c4bf1be2eadea8ad0dca6277c0e4f not found: ID does not exist" Jul 02 13:47:13.207123 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.207029 2561 scope.go:117] "RemoveContainer" containerID="014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c" Jul 02 13:47:13.207228 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.207207 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c"} err="failed to get container status \"014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c\": rpc error: code = NotFound desc = could not find container \"014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c\": container with ID starting with 014a4081c16d3f353565892aed63683e546612d697527a0411849915c0b4dc9c not found: ID does not exist" Jul 02 13:47:13.207265 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.207229 2561 scope.go:117] "RemoveContainer" containerID="880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a" Jul 02 13:47:13.207422 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.207405 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a"} err="failed to get container status \"880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a\": rpc error: code = NotFound desc = could not find container \"880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a\": container with ID starting with 880812bd5ab2f954810f59b8af680aec466c328a14d0f6855a55313a0a69b76a not found: ID does not exist" Jul 02 13:47:13.247994 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.247970 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:47:13.248273 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248261 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ffe8e9f-a3ee-4370-a09e-60312179e091" containerName="console" Jul 02 13:47:13.248330 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248275 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ffe8e9f-a3ee-4370-a09e-60312179e091" containerName="console" Jul 02 13:47:13.248330 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248283 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="prom-label-proxy" Jul 02 13:47:13.248330 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248290 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="prom-label-proxy" Jul 02 13:47:13.248330 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248307 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="alertmanager" Jul 02 13:47:13.248330 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248312 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="alertmanager" Jul 02 13:47:13.248330 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248319 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy-web" Jul 02 13:47:13.248330 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248324 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy-web" Jul 02 13:47:13.248330 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248331 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248336 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248343 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="config-reloader" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248348 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="config-reloader" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248354 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="init-config-reloader" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248358 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="init-config-reloader" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248367 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" containerName="registry" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248372 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" containerName="registry" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248378 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy-metric" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248382 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy-metric" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248431 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="alertmanager" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248439 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248446 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="config-reloader" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248452 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy-metric" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248457 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="prom-label-proxy" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248462 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3904b8e4-e2d9-41e1-a5c9-2021f5afb23a" containerName="registry" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248467 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ffe8e9f-a3ee-4370-a09e-60312179e091" containerName="console" Jul 02 13:47:13.248587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.248471 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" containerName="kube-rbac-proxy-web" Jul 02 13:47:13.251919 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.251905 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.257607 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.257585 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 13:47:13.257847 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.257830 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 13:47:13.257847 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.257838 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 13:47:13.257847 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.257847 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 13:47:13.258831 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.258787 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rxvbr\"" Jul 02 13:47:13.259138 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.259119 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 13:47:13.259237 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.259136 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 13:47:13.259292 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.259242 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 13:47:13.260224 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.260207 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 13:47:13.265073 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265028 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265177 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265177 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265121 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/29359fc4-11ce-4837-bedf-96f78dfe18e0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265177 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265153 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29359fc4-11ce-4837-bedf-96f78dfe18e0-config-out\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265339 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265190 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29359fc4-11ce-4837-bedf-96f78dfe18e0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265339 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265213 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-config-volume\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265339 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265271 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265339 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265310 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265339 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265329 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265598 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265348 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-web-config\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265598 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265366 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29359fc4-11ce-4837-bedf-96f78dfe18e0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265598 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265387 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29359fc4-11ce-4837-bedf-96f78dfe18e0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.265598 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.265476 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdf2l\" (UniqueName: \"kubernetes.io/projected/29359fc4-11ce-4837-bedf-96f78dfe18e0-kube-api-access-rdf2l\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.266566 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.266544 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 13:47:13.272852 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.272832 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:47:13.366505 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.366468 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29359fc4-11ce-4837-bedf-96f78dfe18e0-config-out\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.366505 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.366510 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29359fc4-11ce-4837-bedf-96f78dfe18e0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.366746 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.366537 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-config-volume\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.366746 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.366710 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.366844 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.366751 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.366844 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.366773 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.366844 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.366805 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-web-config\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.366844 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.366826 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29359fc4-11ce-4837-bedf-96f78dfe18e0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.367082 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.366956 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29359fc4-11ce-4837-bedf-96f78dfe18e0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.367082 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.367025 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdf2l\" (UniqueName: \"kubernetes.io/projected/29359fc4-11ce-4837-bedf-96f78dfe18e0-kube-api-access-rdf2l\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.367196 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.367085 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.367196 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.367129 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.367196 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.367155 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/29359fc4-11ce-4837-bedf-96f78dfe18e0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.367488 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.367461 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/29359fc4-11ce-4837-bedf-96f78dfe18e0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.367822 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.367797 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29359fc4-11ce-4837-bedf-96f78dfe18e0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.368514 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.368493 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29359fc4-11ce-4837-bedf-96f78dfe18e0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.369545 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.369518 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29359fc4-11ce-4837-bedf-96f78dfe18e0-config-out\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.369721 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.369698 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.369774 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.369722 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-config-volume\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.370220 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.370198 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.370518 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.370498 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-web-config\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.370603 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.370573 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29359fc4-11ce-4837-bedf-96f78dfe18e0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.370659 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.370627 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.371139 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.371122 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.371733 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.371716 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29359fc4-11ce-4837-bedf-96f78dfe18e0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.388702 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.388676 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdf2l\" (UniqueName: \"kubernetes.io/projected/29359fc4-11ce-4837-bedf-96f78dfe18e0-kube-api-access-rdf2l\") pod \"alertmanager-main-0\" (UID: \"29359fc4-11ce-4837-bedf-96f78dfe18e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.561260 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.561226 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:47:13.723436 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:13.723328 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:47:13.728213 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:47:13.728187 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29359fc4_11ce_4837_bedf_96f78dfe18e0.slice/crio-61b020c2ffc3d010aad1d9dd0e23969ee4ddd1202649320f1260ea40627f5cc4 WatchSource:0}: Error finding container 61b020c2ffc3d010aad1d9dd0e23969ee4ddd1202649320f1260ea40627f5cc4: Status 404 returned error can't find the container with id 61b020c2ffc3d010aad1d9dd0e23969ee4ddd1202649320f1260ea40627f5cc4 Jul 02 13:47:14.086294 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:14.086261 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19599919-bdf9-4801-9b53-41e23dc4fdc2" path="/var/lib/kubelet/pods/19599919-bdf9-4801-9b53-41e23dc4fdc2/volumes" Jul 02 13:47:14.157593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:14.157557 2561 generic.go:356] "Generic (PLEG): container finished" podID="29359fc4-11ce-4837-bedf-96f78dfe18e0" containerID="b06318f06433a4ea672d30d61deca23bd22414a7caa405d6d3f7b6ce500d97b5" exitCode=0 Jul 02 13:47:14.157593 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:14.157597 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29359fc4-11ce-4837-bedf-96f78dfe18e0","Type":"ContainerDied","Data":"b06318f06433a4ea672d30d61deca23bd22414a7caa405d6d3f7b6ce500d97b5"} Jul 02 13:47:14.157937 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:14.157660 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29359fc4-11ce-4837-bedf-96f78dfe18e0","Type":"ContainerStarted","Data":"61b020c2ffc3d010aad1d9dd0e23969ee4ddd1202649320f1260ea40627f5cc4"} Jul 02 13:47:15.165788 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.165751 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29359fc4-11ce-4837-bedf-96f78dfe18e0","Type":"ContainerStarted","Data":"5baa4abacbd2f58e3c3d11702cf5d197f5bf7e9b645f24f69c4b9e7c318908dc"} Jul 02 13:47:15.166190 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.165841 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29359fc4-11ce-4837-bedf-96f78dfe18e0","Type":"ContainerStarted","Data":"c0d8fe527af9093c6d298ec7543224fa81fec3104500de24efb67d291c16684d"} Jul 02 13:47:15.166190 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.165867 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29359fc4-11ce-4837-bedf-96f78dfe18e0","Type":"ContainerStarted","Data":"100fc6d384a7adc4b6a504285214615920272c54db2235ebdfcf044fe424a784"} Jul 02 13:47:15.166190 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.165893 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29359fc4-11ce-4837-bedf-96f78dfe18e0","Type":"ContainerStarted","Data":"6596ccb03649079c3702a503bdbc0d98b6abed7145ed10da897ffcd0df22008b"} Jul 02 13:47:15.166190 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.165916 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29359fc4-11ce-4837-bedf-96f78dfe18e0","Type":"ContainerStarted","Data":"885a54b09a8a576a7ab72b54cda12daad5c10c74133efe44ef1c39efcda488f4"} Jul 02 13:47:15.166190 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.165940 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29359fc4-11ce-4837-bedf-96f78dfe18e0","Type":"ContainerStarted","Data":"52e66d956ef8d727fcdf7b0bcea19db1c0bb57476f50510759dafd69f9df0afb"} Jul 02 13:47:15.676572 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.676509 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.661207386 podStartE2EDuration="2.676491874s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.284488ms" isStatefulPod=false podCreationTimestamp="2026-07-02 13:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:15.205273142 +0000 UTC m=+201.531834200" watchObservedRunningTime="2026-07-02 13:47:15.676491874 +0000 UTC m=+202.003052950" Jul 02 13:47:15.677196 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.677173 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:47:15.677522 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.677485 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="prometheus" containerID="cri-o://420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3" gracePeriod=600 Jul 02 13:47:15.677522 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.677501 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy" containerID="cri-o://a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d" gracePeriod=600 Jul 02 13:47:15.677700 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.677527 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy-web" containerID="cri-o://7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46" gracePeriod=600 Jul 02 13:47:15.677700 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.677532 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="config-reloader" containerID="cri-o://61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df" gracePeriod=600 Jul 02 13:47:15.677700 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.677580 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy-thanos" containerID="cri-o://4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e" gracePeriod=600 Jul 02 13:47:15.677700 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.677506 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="thanos-sidecar" containerID="cri-o://84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec" gracePeriod=600 Jul 02 13:47:15.909861 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.909837 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:15.987568 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.987541 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-web-config\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.987742 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.987580 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-kube-rbac-proxy\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.987742 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.987602 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86ljs\" (UniqueName: \"kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-kube-api-access-86ljs\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.987742 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.987632 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-metrics-client-ca\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.987907 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.987756 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.987907 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.987814 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-1\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.987907 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.987846 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-tls-assets\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.987907 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.987872 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-thanos-prometheus-http-client-file\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988142 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.987910 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-2\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988142 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.987945 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-serving-certs-ca-bundle\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988142 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.987971 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-db\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988142 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988007 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-grpc-tls\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988142 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988003 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:47:15.988142 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988066 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-tls\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988142 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988097 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-kubelet-serving-ca-bundle\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988142 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988127 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config-out\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988538 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988171 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-0\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988538 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988197 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988538 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988236 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-metrics-client-certs\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988538 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988278 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988538 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988305 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-trusted-ca-bundle\") pod \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\" (UID: \"b98c100f-9a2d-4141-9938-2c08bfdd48a9\") " Jul 02 13:47:15.988538 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988364 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:47:15.988920 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988650 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-metrics-client-ca\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:15.988920 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.988671 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:15.989030 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.989008 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:47:15.989118 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.989017 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:47:15.989443 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.989418 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:47:15.991551 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.990484 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:15.991551 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.990575 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-kube-api-access-86ljs" (OuterVolumeSpecName: "kube-api-access-86ljs") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "kube-api-access-86ljs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:47:15.991551 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.990678 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:47:15.991551 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.990750 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:47:15.991551 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.990922 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config" (OuterVolumeSpecName: "config") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:15.991844 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.991752 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:15.992265 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.991976 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:47:15.992426 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.992386 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:47:15.993219 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.993188 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:15.993302 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.993212 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:15.993371 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.993317 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config-out" (OuterVolumeSpecName: "config-out") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:47:15.993433 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.993370 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:15.993591 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.993571 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:15.993995 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:15.993974 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:16.002920 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.002895 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-web-config" (OuterVolumeSpecName: "web-config") pod "b98c100f-9a2d-4141-9938-2c08bfdd48a9" (UID: "b98c100f-9a2d-4141-9938-2c08bfdd48a9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:47:16.089462 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089429 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089462 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089463 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089618 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089481 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-web-config\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089618 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089496 2561 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-kube-rbac-proxy\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089618 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089510 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86ljs\" (UniqueName: \"kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-kube-api-access-86ljs\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089618 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089523 2561 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089618 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089536 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b98c100f-9a2d-4141-9938-2c08bfdd48a9-tls-assets\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089618 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089551 2561 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089618 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089565 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089618 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089580 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089618 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089594 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-db\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089618 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089606 2561 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-grpc-tls\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089909 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089621 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089909 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089637 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089909 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089651 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b98c100f-9a2d-4141-9938-2c08bfdd48a9-config-out\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089909 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089665 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b98c100f-9a2d-4141-9938-2c08bfdd48a9-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089909 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089678 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.089909 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.089692 2561 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b98c100f-9a2d-4141-9938-2c08bfdd48a9-secret-metrics-client-certs\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:47:16.173103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173072 2561 generic.go:356] "Generic (PLEG): container finished" podID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerID="4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e" exitCode=0 Jul 02 13:47:16.173103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173099 2561 generic.go:356] "Generic (PLEG): container finished" podID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerID="a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d" exitCode=0 Jul 02 13:47:16.173103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173106 2561 generic.go:356] "Generic (PLEG): container finished" podID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerID="7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46" exitCode=0 Jul 02 13:47:16.173103 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173112 2561 generic.go:356] "Generic (PLEG): container finished" podID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerID="84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec" exitCode=0 Jul 02 13:47:16.173632 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173118 2561 generic.go:356] "Generic (PLEG): container finished" podID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerID="61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df" exitCode=0 Jul 02 13:47:16.173632 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173124 2561 generic.go:356] "Generic (PLEG): container finished" podID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerID="420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3" exitCode=0 Jul 02 13:47:16.173632 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173161 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.173632 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173163 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerDied","Data":"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e"} Jul 02 13:47:16.173632 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173240 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerDied","Data":"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d"} Jul 02 13:47:16.173632 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173281 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerDied","Data":"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46"} Jul 02 13:47:16.173632 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173307 2561 scope.go:117] "RemoveContainer" containerID="4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e" Jul 02 13:47:16.173632 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173325 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerDied","Data":"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec"} Jul 02 13:47:16.173632 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173366 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerDied","Data":"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df"} Jul 02 13:47:16.173632 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173407 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerDied","Data":"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3"} Jul 02 13:47:16.173632 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.173448 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b98c100f-9a2d-4141-9938-2c08bfdd48a9","Type":"ContainerDied","Data":"1d0699c2161f64cf101c645d89738b64c92ddd6d5f01c88f91ea6609f152a329"} Jul 02 13:47:16.182958 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.182903 2561 scope.go:117] "RemoveContainer" containerID="a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d" Jul 02 13:47:16.190011 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.189994 2561 scope.go:117] "RemoveContainer" containerID="7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46" Jul 02 13:47:16.197376 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.197356 2561 scope.go:117] "RemoveContainer" containerID="84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec" Jul 02 13:47:16.197452 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.197375 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:47:16.203890 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.203871 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:47:16.204439 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.204425 2561 scope.go:117] "RemoveContainer" containerID="61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df" Jul 02 13:47:16.210979 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.210960 2561 scope.go:117] "RemoveContainer" containerID="420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3" Jul 02 13:47:16.217790 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.217773 2561 scope.go:117] "RemoveContainer" containerID="e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070" Jul 02 13:47:16.224124 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.224110 2561 scope.go:117] "RemoveContainer" containerID="4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e" Jul 02 13:47:16.224366 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:16.224349 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": container with ID starting with 4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e not found: ID does not exist" containerID="4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e" Jul 02 13:47:16.224437 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.224376 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e"} err="failed to get container status \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": rpc error: code = NotFound desc = could not find container \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": container with ID starting with 4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e not found: ID does not exist" Jul 02 13:47:16.224437 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.224400 2561 scope.go:117] "RemoveContainer" containerID="a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d" Jul 02 13:47:16.224661 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:16.224643 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": container with ID starting with a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d not found: ID does not exist" containerID="a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d" Jul 02 13:47:16.224720 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.224670 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d"} err="failed to get container status \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": rpc error: code = NotFound desc = could not find container \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": container with ID starting with a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d not found: ID does not exist" Jul 02 13:47:16.224720 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.224690 2561 scope.go:117] "RemoveContainer" containerID="7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46" Jul 02 13:47:16.224904 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:16.224888 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": container with ID starting with 7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46 not found: ID does not exist" containerID="7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46" Jul 02 13:47:16.224943 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.224909 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46"} err="failed to get container status \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": rpc error: code = NotFound desc = could not find container \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": container with ID starting with 7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46 not found: ID does not exist" Jul 02 13:47:16.224943 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.224924 2561 scope.go:117] "RemoveContainer" containerID="84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec" Jul 02 13:47:16.225114 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:16.225098 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": container with ID starting with 84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec not found: ID does not exist" containerID="84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec" Jul 02 13:47:16.225167 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.225118 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec"} err="failed to get container status \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": rpc error: code = NotFound desc = could not find container \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": container with ID starting with 84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec not found: ID does not exist" Jul 02 13:47:16.225167 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.225131 2561 scope.go:117] "RemoveContainer" containerID="61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df" Jul 02 13:47:16.225330 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:16.225316 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": container with ID starting with 61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df not found: ID does not exist" containerID="61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df" Jul 02 13:47:16.225372 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.225332 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df"} err="failed to get container status \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": rpc error: code = NotFound desc = could not find container \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": container with ID starting with 61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df not found: ID does not exist" Jul 02 13:47:16.225372 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.225343 2561 scope.go:117] "RemoveContainer" containerID="420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3" Jul 02 13:47:16.225547 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:16.225533 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": container with ID starting with 420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3 not found: ID does not exist" containerID="420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3" Jul 02 13:47:16.225587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.225550 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3"} err="failed to get container status \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": rpc error: code = NotFound desc = could not find container \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": container with ID starting with 420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3 not found: ID does not exist" Jul 02 13:47:16.225587 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.225563 2561 scope.go:117] "RemoveContainer" containerID="e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070" Jul 02 13:47:16.225781 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:47:16.225765 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": container with ID starting with e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070 not found: ID does not exist" containerID="e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070" Jul 02 13:47:16.225820 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.225794 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070"} err="failed to get container status \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": rpc error: code = NotFound desc = could not find container \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": container with ID starting with e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070 not found: ID does not exist" Jul 02 13:47:16.225820 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.225808 2561 scope.go:117] "RemoveContainer" containerID="4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e" Jul 02 13:47:16.226014 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.225995 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e"} err="failed to get container status \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": rpc error: code = NotFound desc = could not find container \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": container with ID starting with 4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e not found: ID does not exist" Jul 02 13:47:16.226077 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.226015 2561 scope.go:117] "RemoveContainer" containerID="a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d" Jul 02 13:47:16.226290 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.226263 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d"} err="failed to get container status \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": rpc error: code = NotFound desc = could not find container \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": container with ID starting with a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d not found: ID does not exist" Jul 02 13:47:16.226290 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.226281 2561 scope.go:117] "RemoveContainer" containerID="7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46" Jul 02 13:47:16.226502 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.226487 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46"} err="failed to get container status \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": rpc error: code = NotFound desc = could not find container \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": container with ID starting with 7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46 not found: ID does not exist" Jul 02 13:47:16.226545 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.226502 2561 scope.go:117] "RemoveContainer" containerID="84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec" Jul 02 13:47:16.226691 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.226660 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec"} err="failed to get container status \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": rpc error: code = NotFound desc = could not find container \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": container with ID starting with 84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec not found: ID does not exist" Jul 02 13:47:16.226691 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.226681 2561 scope.go:117] "RemoveContainer" containerID="61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df" Jul 02 13:47:16.226906 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.226890 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df"} err="failed to get container status \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": rpc error: code = NotFound desc = could not find container \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": container with ID starting with 61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df not found: ID does not exist" Jul 02 13:47:16.226969 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.226906 2561 scope.go:117] "RemoveContainer" containerID="420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3" Jul 02 13:47:16.227147 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.227124 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3"} err="failed to get container status \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": rpc error: code = NotFound desc = could not find container \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": container with ID starting with 420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3 not found: ID does not exist" Jul 02 13:47:16.227147 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.227146 2561 scope.go:117] "RemoveContainer" containerID="e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070" Jul 02 13:47:16.227354 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.227321 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070"} err="failed to get container status \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": rpc error: code = NotFound desc = could not find container \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": container with ID starting with e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070 not found: ID does not exist" Jul 02 13:47:16.227354 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.227344 2561 scope.go:117] "RemoveContainer" containerID="4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e" Jul 02 13:47:16.227642 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.227620 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e"} err="failed to get container status \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": rpc error: code = NotFound desc = could not find container \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": container with ID starting with 4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e not found: ID does not exist" Jul 02 13:47:16.227752 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.227644 2561 scope.go:117] "RemoveContainer" containerID="a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d" Jul 02 13:47:16.227958 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.227928 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d"} err="failed to get container status \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": rpc error: code = NotFound desc = could not find container \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": container with ID starting with a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d not found: ID does not exist" Jul 02 13:47:16.227958 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.227952 2561 scope.go:117] "RemoveContainer" containerID="7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46" Jul 02 13:47:16.228198 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.228182 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46"} err="failed to get container status \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": rpc error: code = NotFound desc = could not find container \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": container with ID starting with 7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46 not found: ID does not exist" Jul 02 13:47:16.228240 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.228199 2561 scope.go:117] "RemoveContainer" containerID="84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec" Jul 02 13:47:16.228407 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.228385 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec"} err="failed to get container status \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": rpc error: code = NotFound desc = could not find container \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": container with ID starting with 84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec not found: ID does not exist" Jul 02 13:47:16.228445 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.228409 2561 scope.go:117] "RemoveContainer" containerID="61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df" Jul 02 13:47:16.228604 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.228587 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df"} err="failed to get container status \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": rpc error: code = NotFound desc = could not find container \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": container with ID starting with 61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df not found: ID does not exist" Jul 02 13:47:16.228652 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.228604 2561 scope.go:117] "RemoveContainer" containerID="420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3" Jul 02 13:47:16.228802 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.228783 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3"} err="failed to get container status \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": rpc error: code = NotFound desc = could not find container \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": container with ID starting with 420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3 not found: ID does not exist" Jul 02 13:47:16.228848 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.228805 2561 scope.go:117] "RemoveContainer" containerID="e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070" Jul 02 13:47:16.229072 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.229014 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070"} err="failed to get container status \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": rpc error: code = NotFound desc = could not find container \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": container with ID starting with e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070 not found: ID does not exist" Jul 02 13:47:16.229072 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.229066 2561 scope.go:117] "RemoveContainer" containerID="4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e" Jul 02 13:47:16.229272 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.229255 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e"} err="failed to get container status \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": rpc error: code = NotFound desc = could not find container \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": container with ID starting with 4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e not found: ID does not exist" Jul 02 13:47:16.229315 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.229272 2561 scope.go:117] "RemoveContainer" containerID="a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d" Jul 02 13:47:16.229445 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.229430 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d"} err="failed to get container status \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": rpc error: code = NotFound desc = could not find container \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": container with ID starting with a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d not found: ID does not exist" Jul 02 13:47:16.229497 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.229446 2561 scope.go:117] "RemoveContainer" containerID="7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46" Jul 02 13:47:16.229657 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.229638 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46"} err="failed to get container status \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": rpc error: code = NotFound desc = could not find container \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": container with ID starting with 7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46 not found: ID does not exist" Jul 02 13:47:16.229726 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.229659 2561 scope.go:117] "RemoveContainer" containerID="84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec" Jul 02 13:47:16.229864 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.229846 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec"} err="failed to get container status \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": rpc error: code = NotFound desc = could not find container \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": container with ID starting with 84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec not found: ID does not exist" Jul 02 13:47:16.229931 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.229865 2561 scope.go:117] "RemoveContainer" containerID="61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df" Jul 02 13:47:16.230089 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.230072 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df"} err="failed to get container status \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": rpc error: code = NotFound desc = could not find container \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": container with ID starting with 61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df not found: ID does not exist" Jul 02 13:47:16.230143 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.230089 2561 scope.go:117] "RemoveContainer" containerID="420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3" Jul 02 13:47:16.230322 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.230303 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3"} err="failed to get container status \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": rpc error: code = NotFound desc = could not find container \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": container with ID starting with 420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3 not found: ID does not exist" Jul 02 13:47:16.230322 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.230322 2561 scope.go:117] "RemoveContainer" containerID="e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070" Jul 02 13:47:16.230559 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.230545 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070"} err="failed to get container status \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": rpc error: code = NotFound desc = could not find container \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": container with ID starting with e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070 not found: ID does not exist" Jul 02 13:47:16.230559 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.230558 2561 scope.go:117] "RemoveContainer" containerID="4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e" Jul 02 13:47:16.230749 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.230733 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e"} err="failed to get container status \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": rpc error: code = NotFound desc = could not find container \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": container with ID starting with 4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e not found: ID does not exist" Jul 02 13:47:16.230749 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.230748 2561 scope.go:117] "RemoveContainer" containerID="a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d" Jul 02 13:47:16.230970 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.230952 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d"} err="failed to get container status \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": rpc error: code = NotFound desc = could not find container \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": container with ID starting with a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d not found: ID does not exist" Jul 02 13:47:16.231015 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.230970 2561 scope.go:117] "RemoveContainer" containerID="7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46" Jul 02 13:47:16.231188 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.231172 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46"} err="failed to get container status \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": rpc error: code = NotFound desc = could not find container \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": container with ID starting with 7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46 not found: ID does not exist" Jul 02 13:47:16.231188 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.231186 2561 scope.go:117] "RemoveContainer" containerID="84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec" Jul 02 13:47:16.231387 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.231370 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec"} err="failed to get container status \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": rpc error: code = NotFound desc = could not find container \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": container with ID starting with 84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec not found: ID does not exist" Jul 02 13:47:16.231437 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.231387 2561 scope.go:117] "RemoveContainer" containerID="61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df" Jul 02 13:47:16.231573 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.231558 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df"} err="failed to get container status \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": rpc error: code = NotFound desc = could not find container \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": container with ID starting with 61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df not found: ID does not exist" Jul 02 13:47:16.231617 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.231573 2561 scope.go:117] "RemoveContainer" containerID="420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3" Jul 02 13:47:16.231747 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.231731 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3"} err="failed to get container status \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": rpc error: code = NotFound desc = could not find container \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": container with ID starting with 420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3 not found: ID does not exist" Jul 02 13:47:16.231792 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.231747 2561 scope.go:117] "RemoveContainer" containerID="e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070" Jul 02 13:47:16.231927 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.231909 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070"} err="failed to get container status \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": rpc error: code = NotFound desc = could not find container \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": container with ID starting with e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070 not found: ID does not exist" Jul 02 13:47:16.231962 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.231927 2561 scope.go:117] "RemoveContainer" containerID="4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e" Jul 02 13:47:16.232159 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.232144 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e"} err="failed to get container status \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": rpc error: code = NotFound desc = could not find container \"4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e\": container with ID starting with 4e87fac027a33e4ae7c7848eb39713ec394adb3e566490515040b4779f735c8e not found: ID does not exist" Jul 02 13:47:16.232208 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.232159 2561 scope.go:117] "RemoveContainer" containerID="a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d" Jul 02 13:47:16.232352 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.232334 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d"} err="failed to get container status \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": rpc error: code = NotFound desc = could not find container \"a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d\": container with ID starting with a39269defb3e0acdff68e1cca2316c6445f26ce9c57840ca4d1e2da041587b9d not found: ID does not exist" Jul 02 13:47:16.232396 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.232352 2561 scope.go:117] "RemoveContainer" containerID="7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46" Jul 02 13:47:16.232556 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.232540 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46"} err="failed to get container status \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": rpc error: code = NotFound desc = could not find container \"7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46\": container with ID starting with 7d7d7d5578a77dffd49975acc192a7c87ddc3f48d572edfe53c678885bfa2b46 not found: ID does not exist" Jul 02 13:47:16.232595 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.232556 2561 scope.go:117] "RemoveContainer" containerID="84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec" Jul 02 13:47:16.232758 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.232740 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec"} err="failed to get container status \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": rpc error: code = NotFound desc = could not find container \"84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec\": container with ID starting with 84758b1d8acd4cbfed132a15cdbe811a5f0f0133abb3608f896e5538d1dcefec not found: ID does not exist" Jul 02 13:47:16.232798 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.232758 2561 scope.go:117] "RemoveContainer" containerID="61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df" Jul 02 13:47:16.232954 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.232936 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df"} err="failed to get container status \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": rpc error: code = NotFound desc = could not find container \"61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df\": container with ID starting with 61ed6034549481200890678094db941ac4b0400cf0d250e19af9b95d695d14df not found: ID does not exist" Jul 02 13:47:16.233003 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.232953 2561 scope.go:117] "RemoveContainer" containerID="420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3" Jul 02 13:47:16.233176 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.233161 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3"} err="failed to get container status \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": rpc error: code = NotFound desc = could not find container \"420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3\": container with ID starting with 420f12333eba43d1e7833210b75b19e6ed3cd6e761e537b411f9ad7c34eef0a3 not found: ID does not exist" Jul 02 13:47:16.233237 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.233176 2561 scope.go:117] "RemoveContainer" containerID="e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070" Jul 02 13:47:16.233393 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.233376 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070"} err="failed to get container status \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": rpc error: code = NotFound desc = could not find container \"e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070\": container with ID starting with e14d903cfb6987fe33120916959c29871c8acf9056cfeab2c4da0cd70e12d070 not found: ID does not exist" Jul 02 13:47:16.247443 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247387 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:47:16.247758 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247741 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy-web" Jul 02 13:47:16.247828 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247760 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy-web" Jul 02 13:47:16.247828 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247773 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="config-reloader" Jul 02 13:47:16.247828 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247782 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="config-reloader" Jul 02 13:47:16.247828 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247799 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="prometheus" Jul 02 13:47:16.247828 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247807 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="prometheus" Jul 02 13:47:16.247828 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247818 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="thanos-sidecar" Jul 02 13:47:16.247828 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247826 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="thanos-sidecar" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247841 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="init-config-reloader" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247849 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="init-config-reloader" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247863 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247871 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247888 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy-thanos" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247896 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy-thanos" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247969 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="prometheus" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247981 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="thanos-sidecar" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247990 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.247999 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy-thanos" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.248007 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="kube-rbac-proxy-web" Jul 02 13:47:16.248197 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.248015 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" containerName="config-reloader" Jul 02 13:47:16.252644 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.252624 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.255856 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.255835 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 13:47:16.256009 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.255990 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vvn4n\"" Jul 02 13:47:16.256342 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.256328 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 13:47:16.257476 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.257449 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 13:47:16.257476 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.257478 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 13:47:16.257683 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.257458 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 13:47:16.257968 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.257840 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 13:47:16.257968 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.257848 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 13:47:16.257968 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.257862 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 13:47:16.257968 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.257855 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 13:47:16.258183 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.258072 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-alg4p786vclg0\"" Jul 02 13:47:16.258620 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.258606 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 13:47:16.258620 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.258617 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 13:47:16.258754 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.258663 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 13:47:16.261290 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.261259 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 13:47:16.265578 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.265560 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 13:47:16.271970 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.271949 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:47:16.291597 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291577 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.291688 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291606 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.291688 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291632 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/420157d1-d7e5-401e-a1a8-adf21ae0465d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.291688 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291650 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.291803 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291730 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.291803 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291761 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.291803 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291785 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.291896 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291807 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.291896 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291835 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.291896 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291857 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.291896 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291876 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.291896 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291890 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.292102 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291918 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsb89\" (UniqueName: \"kubernetes.io/projected/420157d1-d7e5-401e-a1a8-adf21ae0465d-kube-api-access-gsb89\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.292102 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291936 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-config\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.292102 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291977 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.292102 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.291997 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-web-config\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.292102 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.292015 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.292102 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.292080 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/420157d1-d7e5-401e-a1a8-adf21ae0465d-config-out\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.292102 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.292097 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.292299 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.292119 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.392719 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.392676 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.392719 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.392722 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.392962 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.392748 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/420157d1-d7e5-401e-a1a8-adf21ae0465d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.392962 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.392771 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.392962 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.392810 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.392962 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.392831 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.392962 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.392863 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.392962 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.392887 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.392962 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.392917 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.392962 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.392951 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.392985 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.393011 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.393076 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsb89\" (UniqueName: \"kubernetes.io/projected/420157d1-d7e5-401e-a1a8-adf21ae0465d-kube-api-access-gsb89\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.393103 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-config\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.393138 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.393161 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-web-config\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.393185 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.393232 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/420157d1-d7e5-401e-a1a8-adf21ae0465d-config-out\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.393257 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393366 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.393297 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393840 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.393600 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.393840 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.393793 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.394097 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.394073 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.394990 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.394959 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.395325 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.395298 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.395831 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.395803 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.395911 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.395829 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.396242 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.396212 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/420157d1-d7e5-401e-a1a8-adf21ae0465d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.396364 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.396337 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.396427 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.396338 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.396745 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.396721 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-web-config\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.397214 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.397188 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.397357 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.397311 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/420157d1-d7e5-401e-a1a8-adf21ae0465d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.397640 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.397619 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.397884 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.397867 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/420157d1-d7e5-401e-a1a8-adf21ae0465d-config-out\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.398539 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.398515 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.398622 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.398552 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.398779 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.398758 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-config\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.398854 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.398838 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/420157d1-d7e5-401e-a1a8-adf21ae0465d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.415400 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.415381 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsb89\" (UniqueName: \"kubernetes.io/projected/420157d1-d7e5-401e-a1a8-adf21ae0465d-kube-api-access-gsb89\") pod \"prometheus-k8s-0\" (UID: \"420157d1-d7e5-401e-a1a8-adf21ae0465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.564142 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.564021 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:16.704271 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:16.704243 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:47:16.707183 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:47:16.707142 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420157d1_d7e5_401e_a1a8_adf21ae0465d.slice/crio-77922dd43cac02906d7c322c168f54f3f3aa1653617e6ee3cb03a7c51061dfae WatchSource:0}: Error finding container 77922dd43cac02906d7c322c168f54f3f3aa1653617e6ee3cb03a7c51061dfae: Status 404 returned error can't find the container with id 77922dd43cac02906d7c322c168f54f3f3aa1653617e6ee3cb03a7c51061dfae Jul 02 13:47:17.179239 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:17.179206 2561 generic.go:356] "Generic (PLEG): container finished" podID="420157d1-d7e5-401e-a1a8-adf21ae0465d" containerID="9c976ea5220d637f0240f4ebe51929167f7b98d5584dacb358d7fc7210c98f5e" exitCode=0 Jul 02 13:47:17.179634 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:17.179293 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"420157d1-d7e5-401e-a1a8-adf21ae0465d","Type":"ContainerDied","Data":"9c976ea5220d637f0240f4ebe51929167f7b98d5584dacb358d7fc7210c98f5e"} Jul 02 13:47:17.179634 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:17.179376 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"420157d1-d7e5-401e-a1a8-adf21ae0465d","Type":"ContainerStarted","Data":"77922dd43cac02906d7c322c168f54f3f3aa1653617e6ee3cb03a7c51061dfae"} Jul 02 13:47:18.086559 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:18.086522 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b98c100f-9a2d-4141-9938-2c08bfdd48a9" path="/var/lib/kubelet/pods/b98c100f-9a2d-4141-9938-2c08bfdd48a9/volumes" Jul 02 13:47:18.191821 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:18.191785 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"420157d1-d7e5-401e-a1a8-adf21ae0465d","Type":"ContainerStarted","Data":"74f14201c5462c162d5dc4fa1e9c5f79fb41d9dce8a41955f212c3891066b958"} Jul 02 13:47:18.192217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:18.191879 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"420157d1-d7e5-401e-a1a8-adf21ae0465d","Type":"ContainerStarted","Data":"b5e84bc0b967cee64eb06d99fad650f649cf8d85958223ef23ee59edcf6d93f1"} Jul 02 13:47:18.192217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:18.191907 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"420157d1-d7e5-401e-a1a8-adf21ae0465d","Type":"ContainerStarted","Data":"94b3ca99f36b709beb1d858cdb7f95029700c4a4269d3036ffb92d2e49a9c0b5"} Jul 02 13:47:18.192217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:18.191932 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"420157d1-d7e5-401e-a1a8-adf21ae0465d","Type":"ContainerStarted","Data":"fc1b58368efe5b0a6a4797a013f907e102a4af24bf6cda190c0869158675e731"} Jul 02 13:47:18.192217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:18.191955 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"420157d1-d7e5-401e-a1a8-adf21ae0465d","Type":"ContainerStarted","Data":"a96f925508c35cb70c47605550866380534c1c3d8dbfd9a8fae7acc8e1eb1d95"} Jul 02 13:47:18.192217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:18.191982 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"420157d1-d7e5-401e-a1a8-adf21ae0465d","Type":"ContainerStarted","Data":"67d1322216475e35e158987433637f7a48fdb98770b2b1dfd437ae1365a783e5"} Jul 02 13:47:18.237444 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:18.237389 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.220481734 podStartE2EDuration="2.237368951s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.887217ms" isStatefulPod=false podCreationTimestamp="2026-07-02 13:47:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:18.231803853 +0000 UTC m=+204.558364910" watchObservedRunningTime="2026-07-02 13:47:18.237368951 +0000 UTC m=+204.563930007" Jul 02 13:47:18.975864 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:18.975828 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:19.076950 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:19.076920 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:20.390638 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:20.390590 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:20.491087 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:20.491015 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:20.552669 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:20.552630 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:21.564768 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:21.564727 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:47:35.891811 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:35.891773 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:35.993879 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:35.993839 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:55.836380 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:55.836329 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:47:55.917436 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:47:55.917401 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:48:16.564579 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:48:16.564487 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:48:16.580273 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:48:16.580239 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:48:17.441797 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:48:17.441770 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:48:31.613645 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:48:31.613612 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:48:54.031324 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:48:54.031294 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 13:48:54.036062 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:48:54.035357 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 13:48:54.044169 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:48:54.044148 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 13:48:59.356626 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:48:59.356591 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:48:59.803735 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:48:59.803699 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:48:59.884651 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:48:59.884620 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:49:00.077363 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:49:00.077272 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:49:00.178521 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:49:00.178489 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:49:00.288247 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:49:00.288212 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:49:22.384974 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:49:22.384934 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:49:22.477498 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:49:22.477462 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:49:33.697926 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:49:33.697892 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:49:33.930221 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:49:33.930186 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:49:34.075855 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:49:34.075816 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:49:35.996395 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:49:35.996361 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:49:46.756868 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:49:46.756783 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:49:46.844296 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:49:46.844259 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:50:42.754264 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:50:42.754231 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:50:42.845033 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:50:42.844998 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:50:42.930496 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:50:42.930447 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:50:43.030539 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:50:43.030453 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:50:43.221504 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:50:43.221464 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:50:43.310313 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:50:43.310231 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:50:43.371712 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:50:43.371678 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:51:02.411755 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:02.411719 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:51:13.285257 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:13.285216 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:51:13.349281 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:13.349246 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:51:13.408522 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:13.408485 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:51:53.147861 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.147811 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:51:53.209598 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.209563 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-6b2rb"] Jul 02 13:51:53.213271 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.213252 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-6b2rb" Jul 02 13:51:53.215830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.215794 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 02 13:51:53.215970 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.215856 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 02 13:51:53.216451 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.216429 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Jul 02 13:51:53.216529 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.216457 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-5c8tc\"" Jul 02 13:51:53.221073 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.221027 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-6b2rb"] Jul 02 13:51:53.231800 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.231771 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:51:53.320217 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.320181 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl5hk\" (UniqueName: \"kubernetes.io/projected/294774ae-0e9f-4b06-ad78-07d5b880c792-kube-api-access-xl5hk\") pod \"odh-model-controller-84b54b7f97-6b2rb\" (UID: \"294774ae-0e9f-4b06-ad78-07d5b880c792\") " pod="kserve/odh-model-controller-84b54b7f97-6b2rb" Jul 02 13:51:53.320417 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.320244 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/294774ae-0e9f-4b06-ad78-07d5b880c792-cert\") pod \"odh-model-controller-84b54b7f97-6b2rb\" (UID: \"294774ae-0e9f-4b06-ad78-07d5b880c792\") " pod="kserve/odh-model-controller-84b54b7f97-6b2rb" Jul 02 13:51:53.372665 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.372624 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:51:53.421149 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.421060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl5hk\" (UniqueName: \"kubernetes.io/projected/294774ae-0e9f-4b06-ad78-07d5b880c792-kube-api-access-xl5hk\") pod \"odh-model-controller-84b54b7f97-6b2rb\" (UID: \"294774ae-0e9f-4b06-ad78-07d5b880c792\") " pod="kserve/odh-model-controller-84b54b7f97-6b2rb" Jul 02 13:51:53.421289 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.421147 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/294774ae-0e9f-4b06-ad78-07d5b880c792-cert\") pod \"odh-model-controller-84b54b7f97-6b2rb\" (UID: \"294774ae-0e9f-4b06-ad78-07d5b880c792\") " pod="kserve/odh-model-controller-84b54b7f97-6b2rb" Jul 02 13:51:53.421335 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:51:53.421287 2561 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 02 13:51:53.421370 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:51:53.421347 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/294774ae-0e9f-4b06-ad78-07d5b880c792-cert podName:294774ae-0e9f-4b06-ad78-07d5b880c792 nodeName:}" failed. No retries permitted until 2026-07-02 13:51:53.921329795 +0000 UTC m=+480.247890834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/294774ae-0e9f-4b06-ad78-07d5b880c792-cert") pod "odh-model-controller-84b54b7f97-6b2rb" (UID: "294774ae-0e9f-4b06-ad78-07d5b880c792") : secret "odh-model-controller-webhook-cert" not found Jul 02 13:51:53.430451 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.430433 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl5hk\" (UniqueName: \"kubernetes.io/projected/294774ae-0e9f-4b06-ad78-07d5b880c792-kube-api-access-xl5hk\") pod \"odh-model-controller-84b54b7f97-6b2rb\" (UID: \"294774ae-0e9f-4b06-ad78-07d5b880c792\") " pod="kserve/odh-model-controller-84b54b7f97-6b2rb" Jul 02 13:51:53.594305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.594269 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:51:53.884266 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.884228 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:51:53.925241 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.925209 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/294774ae-0e9f-4b06-ad78-07d5b880c792-cert\") pod \"odh-model-controller-84b54b7f97-6b2rb\" (UID: \"294774ae-0e9f-4b06-ad78-07d5b880c792\") " pod="kserve/odh-model-controller-84b54b7f97-6b2rb" Jul 02 13:51:53.927702 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.927670 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/294774ae-0e9f-4b06-ad78-07d5b880c792-cert\") pod \"odh-model-controller-84b54b7f97-6b2rb\" (UID: \"294774ae-0e9f-4b06-ad78-07d5b880c792\") " pod="kserve/odh-model-controller-84b54b7f97-6b2rb" Jul 02 13:51:53.985547 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:53.985508 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:51:54.130058 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:54.130012 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-5c8tc\"" Jul 02 13:51:54.137397 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:54.137332 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-6b2rb" Jul 02 13:51:54.270317 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:54.270288 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-6b2rb"] Jul 02 13:51:54.273237 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:51:54.273212 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294774ae_0e9f_4b06_ad78_07d5b880c792.slice/crio-97ad236d15e913254726a4464f271100f25e725bebf2d87f1e34a1b08d38b525 WatchSource:0}: Error finding container 97ad236d15e913254726a4464f271100f25e725bebf2d87f1e34a1b08d38b525: Status 404 returned error can't find the container with id 97ad236d15e913254726a4464f271100f25e725bebf2d87f1e34a1b08d38b525 Jul 02 13:51:54.274553 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:54.274537 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 13:51:55.270074 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:55.270022 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-6b2rb" event={"ID":"294774ae-0e9f-4b06-ad78-07d5b880c792","Type":"ContainerStarted","Data":"97ad236d15e913254726a4464f271100f25e725bebf2d87f1e34a1b08d38b525"} Jul 02 13:51:58.283274 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:58.283231 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-6b2rb" event={"ID":"294774ae-0e9f-4b06-ad78-07d5b880c792","Type":"ContainerStarted","Data":"d599db3cc5e881b2aff8b6fce35c23ec4172044967024fe60d5f8fdc0483d649"} Jul 02 13:51:58.283679 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:58.283298 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-6b2rb" Jul 02 13:51:58.302864 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:51:58.302813 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-6b2rb" podStartSLOduration=2.327580446 podStartE2EDuration="5.302797993s" totalImagesPullingTime="2.975217547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:51:58.301623012 +0000 UTC m=+484.628184068" watchObservedRunningTime="2026-07-02 13:51:58.302797993 +0000 UTC m=+484.629359047" Jul 02 13:52:09.290389 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:09.290358 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-6b2rb" Jul 02 13:52:10.131111 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.131077 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-hgdjc"] Jul 02 13:52:10.134518 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.134501 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-hgdjc" Jul 02 13:52:10.137433 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.137414 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 02 13:52:10.137872 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.137857 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-7wn6m\"" Jul 02 13:52:10.143938 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.143918 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-hgdjc"] Jul 02 13:52:10.265253 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.265217 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d65cn\" (UniqueName: \"kubernetes.io/projected/6b0600b7-b349-464c-9b85-b9963831ad20-kube-api-access-d65cn\") pod \"s3-init-hgdjc\" (UID: \"6b0600b7-b349-464c-9b85-b9963831ad20\") " pod="kserve/s3-init-hgdjc" Jul 02 13:52:10.265427 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.265268 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b0600b7-b349-464c-9b85-b9963831ad20-model-cache\") pod \"s3-init-hgdjc\" (UID: \"6b0600b7-b349-464c-9b85-b9963831ad20\") " pod="kserve/s3-init-hgdjc" Jul 02 13:52:10.366164 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.366110 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d65cn\" (UniqueName: \"kubernetes.io/projected/6b0600b7-b349-464c-9b85-b9963831ad20-kube-api-access-d65cn\") pod \"s3-init-hgdjc\" (UID: \"6b0600b7-b349-464c-9b85-b9963831ad20\") " pod="kserve/s3-init-hgdjc" Jul 02 13:52:10.366642 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.366187 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b0600b7-b349-464c-9b85-b9963831ad20-model-cache\") pod \"s3-init-hgdjc\" (UID: \"6b0600b7-b349-464c-9b85-b9963831ad20\") " pod="kserve/s3-init-hgdjc" Jul 02 13:52:10.366704 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.366635 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b0600b7-b349-464c-9b85-b9963831ad20-model-cache\") pod \"s3-init-hgdjc\" (UID: \"6b0600b7-b349-464c-9b85-b9963831ad20\") " pod="kserve/s3-init-hgdjc" Jul 02 13:52:10.375201 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.375175 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d65cn\" (UniqueName: \"kubernetes.io/projected/6b0600b7-b349-464c-9b85-b9963831ad20-kube-api-access-d65cn\") pod \"s3-init-hgdjc\" (UID: \"6b0600b7-b349-464c-9b85-b9963831ad20\") " pod="kserve/s3-init-hgdjc" Jul 02 13:52:10.457471 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.457367 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-hgdjc" Jul 02 13:52:10.584952 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:10.584927 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-hgdjc"] Jul 02 13:52:10.587276 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:52:10.587246 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b0600b7_b349_464c_9b85_b9963831ad20.slice/crio-03c213edde48639fe3ebcb7491480af512e9fe9b72e31b5df5bcbfcb0ef54915 WatchSource:0}: Error finding container 03c213edde48639fe3ebcb7491480af512e9fe9b72e31b5df5bcbfcb0ef54915: Status 404 returned error can't find the container with id 03c213edde48639fe3ebcb7491480af512e9fe9b72e31b5df5bcbfcb0ef54915 Jul 02 13:52:11.341404 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:11.341371 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-hgdjc" event={"ID":"6b0600b7-b349-464c-9b85-b9963831ad20","Type":"ContainerStarted","Data":"03c213edde48639fe3ebcb7491480af512e9fe9b72e31b5df5bcbfcb0ef54915"} Jul 02 13:52:15.361326 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:15.361289 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-hgdjc" event={"ID":"6b0600b7-b349-464c-9b85-b9963831ad20","Type":"ContainerStarted","Data":"f16b997f9744d33feeb85befb547ed894ad5256a2c72e7fb8870973dfcf1d309"} Jul 02 13:52:17.372628 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:17.372536 2561 generic.go:356] "Generic (PLEG): container finished" podID="6b0600b7-b349-464c-9b85-b9963831ad20" containerID="f16b997f9744d33feeb85befb547ed894ad5256a2c72e7fb8870973dfcf1d309" exitCode=0 Jul 02 13:52:17.372628 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:17.372574 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-hgdjc" event={"ID":"6b0600b7-b349-464c-9b85-b9963831ad20","Type":"ContainerDied","Data":"f16b997f9744d33feeb85befb547ed894ad5256a2c72e7fb8870973dfcf1d309"} Jul 02 13:52:22.398235 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:22.398195 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-hgdjc" event={"ID":"6b0600b7-b349-464c-9b85-b9963831ad20","Type":"ContainerStarted","Data":"47de4d7055df91f47cad3b79907df12a3b33bae1e51766e7246bc151e3d7624f"} Jul 02 13:52:22.416119 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:52:22.416074 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-hgdjc" podStartSLOduration=1.143894441 podStartE2EDuration="12.416056977s" totalImagesPullingTime="9.092322038s" totalInitContainerRuntime="2.179840498s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:52:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:52:22.414678687 +0000 UTC m=+508.741239742" watchObservedRunningTime="2026-07-02 13:52:22.416056977 +0000 UTC m=+508.742618025" Jul 02 13:53:27.667956 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:27.667859 2561 generic.go:356] "Generic (PLEG): container finished" podID="6b0600b7-b349-464c-9b85-b9963831ad20" containerID="47de4d7055df91f47cad3b79907df12a3b33bae1e51766e7246bc151e3d7624f" exitCode=0 Jul 02 13:53:27.667956 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:27.667931 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-hgdjc" event={"ID":"6b0600b7-b349-464c-9b85-b9963831ad20","Type":"ContainerDied","Data":"47de4d7055df91f47cad3b79907df12a3b33bae1e51766e7246bc151e3d7624f"} Jul 02 13:53:28.192666 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:28.192629 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:28.791682 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:28.791654 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-hgdjc" Jul 02 13:53:28.939625 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:28.939541 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b0600b7-b349-464c-9b85-b9963831ad20-model-cache\") pod \"6b0600b7-b349-464c-9b85-b9963831ad20\" (UID: \"6b0600b7-b349-464c-9b85-b9963831ad20\") " Jul 02 13:53:28.939625 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:28.939620 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d65cn\" (UniqueName: \"kubernetes.io/projected/6b0600b7-b349-464c-9b85-b9963831ad20-kube-api-access-d65cn\") pod \"6b0600b7-b349-464c-9b85-b9963831ad20\" (UID: \"6b0600b7-b349-464c-9b85-b9963831ad20\") " Jul 02 13:53:28.941715 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:28.941688 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b0600b7-b349-464c-9b85-b9963831ad20-kube-api-access-d65cn" (OuterVolumeSpecName: "kube-api-access-d65cn") pod "6b0600b7-b349-464c-9b85-b9963831ad20" (UID: "6b0600b7-b349-464c-9b85-b9963831ad20"). InnerVolumeSpecName "kube-api-access-d65cn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:53:28.958714 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:28.958689 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b0600b7-b349-464c-9b85-b9963831ad20-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b0600b7-b349-464c-9b85-b9963831ad20" (UID: "6b0600b7-b349-464c-9b85-b9963831ad20"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:53:29.041194 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:29.041167 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d65cn\" (UniqueName: \"kubernetes.io/projected/6b0600b7-b349-464c-9b85-b9963831ad20-kube-api-access-d65cn\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:53:29.041194 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:29.041191 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b0600b7-b349-464c-9b85-b9963831ad20-model-cache\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:53:29.677616 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:29.677582 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-hgdjc" Jul 02 13:53:29.677784 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:29.677583 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-hgdjc" event={"ID":"6b0600b7-b349-464c-9b85-b9963831ad20","Type":"ContainerDied","Data":"03c213edde48639fe3ebcb7491480af512e9fe9b72e31b5df5bcbfcb0ef54915"} Jul 02 13:53:29.677784 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:29.677678 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03c213edde48639fe3ebcb7491480af512e9fe9b72e31b5df5bcbfcb0ef54915" Jul 02 13:53:30.062547 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:30.062506 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:30.164765 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:30.164726 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:41.364603 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.364569 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:41.617307 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.617197 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:41.694482 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.694424 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:41.695840 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.695813 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f"] Jul 02 13:53:41.696347 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.696331 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b0600b7-b349-464c-9b85-b9963831ad20" containerName="download-hf-model" Jul 02 13:53:41.696399 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.696351 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b0600b7-b349-464c-9b85-b9963831ad20" containerName="download-hf-model" Jul 02 13:53:41.696399 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.696382 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b0600b7-b349-464c-9b85-b9963831ad20" containerName="s3-init" Jul 02 13:53:41.696399 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.696391 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b0600b7-b349-464c-9b85-b9963831ad20" containerName="s3-init" Jul 02 13:53:41.696496 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.696463 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b0600b7-b349-464c-9b85-b9963831ad20" containerName="s3-init" Jul 02 13:53:41.700236 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.700219 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.702759 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.702734 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Jul 02 13:53:41.702940 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.702917 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-kxrcm\"" Jul 02 13:53:41.703125 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.703107 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 02 13:53:41.703270 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.703152 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 02 13:53:41.703596 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.703577 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 02 13:53:41.703689 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.703613 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Jul 02 13:53:41.703748 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.703699 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 02 13:53:41.710704 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.710683 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f"] Jul 02 13:53:41.738396 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.738351 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.738573 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.738418 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq88q\" (UniqueName: \"kubernetes.io/projected/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-api-access-xq88q\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.738573 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.738486 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.738573 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.738557 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.738739 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.738598 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d4dc493-0eab-44a2-b0f9-9cf16928015d-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.770375 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.770340 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:41.839312 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.839273 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.839513 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.839324 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d4dc493-0eab-44a2-b0f9-9cf16928015d-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.839513 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.839361 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.839513 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.839392 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq88q\" (UniqueName: \"kubernetes.io/projected/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-api-access-xq88q\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.839513 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.839429 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.839841 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.839736 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.840083 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.840034 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.840208 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.840190 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.842648 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.842623 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d4dc493-0eab-44a2-b0f9-9cf16928015d-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.850588 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.850562 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq88q\" (UniqueName: \"kubernetes.io/projected/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-api-access-xq88q\") pod \"isvc-sklearn-graph-1-predictor-866d764d98-72g5f\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:41.872194 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.872115 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:41.963685 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.963642 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:41.969513 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.969482 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg"] Jul 02 13:53:41.973544 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.973521 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:41.976022 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.975998 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Jul 02 13:53:41.976404 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.976386 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Jul 02 13:53:41.985390 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:41.985369 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg"] Jul 02 13:53:42.011817 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.011762 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:53:42.041123 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.041089 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm5m8\" (UniqueName: \"kubernetes.io/projected/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-api-access-wm5m8\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.041305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.041186 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.041305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.041238 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.041305 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.041291 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.041471 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.041333 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.091544 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.091509 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:42.108642 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.108611 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw"] Jul 02 13:53:42.112609 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.112582 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.115543 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.115518 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Jul 02 13:53:42.118919 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.118897 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Jul 02 13:53:42.136638 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.136560 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw"] Jul 02 13:53:42.141994 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.141968 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.142168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.142057 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.142168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.142094 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.142168 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.142135 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9792\" (UniqueName: \"kubernetes.io/projected/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-api-access-z9792\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.142390 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.142180 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.142390 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.142209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.142390 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.142237 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.142390 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.142306 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm5m8\" (UniqueName: \"kubernetes.io/projected/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-api-access-wm5m8\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.142390 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.142338 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.142390 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.142376 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.143129 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.142803 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.143129 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.142902 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.143129 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.143023 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.145751 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.145730 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.155765 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.155743 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm5m8\" (UniqueName: \"kubernetes.io/projected/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-api-access-wm5m8\") pod \"isvc-xgboost-graph-predictor-555fc69d56-72pkg\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.183620 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.183595 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f"] Jul 02 13:53:42.185745 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:53:42.185712 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4dc493_0eab_44a2_b0f9_9cf16928015d.slice/crio-c58198e93ccb42bad1832da38a1f0f662e3be1560e5d2352181d3f4c851270dc WatchSource:0}: Error finding container c58198e93ccb42bad1832da38a1f0f662e3be1560e5d2352181d3f4c851270dc: Status 404 returned error can't find the container with id c58198e93ccb42bad1832da38a1f0f662e3be1560e5d2352181d3f4c851270dc Jul 02 13:53:42.242830 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.242798 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.242989 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.242891 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.242989 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.242935 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9792\" (UniqueName: \"kubernetes.io/projected/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-api-access-z9792\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.242989 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.242962 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.243183 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.242991 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.243183 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:53:42.243065 2561 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Jul 02 13:53:42.243183 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:53:42.243131 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-proxy-tls podName:da6457ff-394d-4ad1-9f8f-a0e9e880dcce nodeName:}" failed. No retries permitted until 2026-07-02 13:53:42.743110502 +0000 UTC m=+589.069671574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-proxy-tls") pod "isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" (UID: "da6457ff-394d-4ad1-9f8f-a0e9e880dcce") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Jul 02 13:53:42.243348 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.243232 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.243880 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.243840 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.243998 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.243904 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.245090 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.245057 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:42.264574 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.264550 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9792\" (UniqueName: \"kubernetes.io/projected/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-api-access-z9792\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.285274 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.285246 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:53:42.360613 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.360562 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:53:42.437070 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.436955 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg"] Jul 02 13:53:42.441525 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:53:42.441493 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd724e45d_5e3c_4ac9_bc2d_891d8da92e1b.slice/crio-a1d4fdcf2a08d6f159fc9f6350e72e942dce685634c84153ffc3a1750f644369 WatchSource:0}: Error finding container a1d4fdcf2a08d6f159fc9f6350e72e942dce685634c84153ffc3a1750f644369: Status 404 returned error can't find the container with id a1d4fdcf2a08d6f159fc9f6350e72e942dce685634c84153ffc3a1750f644369 Jul 02 13:53:42.738195 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.737623 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" event={"ID":"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b","Type":"ContainerStarted","Data":"cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d"} Jul 02 13:53:42.738195 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.737700 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" event={"ID":"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b","Type":"ContainerStarted","Data":"a1d4fdcf2a08d6f159fc9f6350e72e942dce685634c84153ffc3a1750f644369"} Jul 02 13:53:42.739554 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.739524 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" event={"ID":"6d4dc493-0eab-44a2-b0f9-9cf16928015d","Type":"ContainerStarted","Data":"be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea"} Jul 02 13:53:42.739694 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.739583 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" event={"ID":"6d4dc493-0eab-44a2-b0f9-9cf16928015d","Type":"ContainerStarted","Data":"c58198e93ccb42bad1832da38a1f0f662e3be1560e5d2352181d3f4c851270dc"} Jul 02 13:53:42.748061 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.748009 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:42.751050 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:42.751011 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:43.024739 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:43.024642 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:53:43.167155 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:43.167128 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw"] Jul 02 13:53:43.169376 ip-10-0-137-40 kubenswrapper[2561]: W0702 13:53:43.169352 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda6457ff_394d_4ad1_9f8f_a0e9e880dcce.slice/crio-86da81173ff481593846ec1cba6cd78434e20ed4b263d9c190711b6e632ea923 WatchSource:0}: Error finding container 86da81173ff481593846ec1cba6cd78434e20ed4b263d9c190711b6e632ea923: Status 404 returned error can't find the container with id 86da81173ff481593846ec1cba6cd78434e20ed4b263d9c190711b6e632ea923 Jul 02 13:53:43.745518 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:43.745475 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" event={"ID":"da6457ff-394d-4ad1-9f8f-a0e9e880dcce","Type":"ContainerStarted","Data":"a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34"} Jul 02 13:53:43.745921 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:43.745556 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" event={"ID":"da6457ff-394d-4ad1-9f8f-a0e9e880dcce","Type":"ContainerStarted","Data":"86da81173ff481593846ec1cba6cd78434e20ed4b263d9c190711b6e632ea923"} Jul 02 13:53:46.759861 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:46.759825 2561 generic.go:356] "Generic (PLEG): container finished" podID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerID="cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d" exitCode=0 Jul 02 13:53:46.760329 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:46.759912 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" event={"ID":"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b","Type":"ContainerDied","Data":"cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d"} Jul 02 13:53:46.761695 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:46.761663 2561 generic.go:356] "Generic (PLEG): container finished" podID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerID="be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea" exitCode=0 Jul 02 13:53:46.761822 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:46.761713 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" event={"ID":"6d4dc493-0eab-44a2-b0f9-9cf16928015d","Type":"ContainerDied","Data":"be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea"} Jul 02 13:53:47.781146 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:47.781107 2561 generic.go:356] "Generic (PLEG): container finished" podID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerID="a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34" exitCode=0 Jul 02 13:53:47.781615 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:53:47.781180 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" event={"ID":"da6457ff-394d-4ad1-9f8f-a0e9e880dcce","Type":"ContainerDied","Data":"a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34"} Jul 02 13:54:15.405700 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:15.405672 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 13:54:15.406160 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:15.406064 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 13:54:15.977150 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:15.977115 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" event={"ID":"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b","Type":"ContainerStarted","Data":"30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee"} Jul 02 13:54:15.978860 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:15.978836 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" event={"ID":"da6457ff-394d-4ad1-9f8f-a0e9e880dcce","Type":"ContainerStarted","Data":"7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8"} Jul 02 13:54:15.980584 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:15.980559 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" event={"ID":"6d4dc493-0eab-44a2-b0f9-9cf16928015d","Type":"ContainerStarted","Data":"1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2"} Jul 02 13:54:18.998253 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:18.998220 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" event={"ID":"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b","Type":"ContainerStarted","Data":"962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845"} Jul 02 13:54:18.998717 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:18.998423 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:54:18.998717 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:18.998484 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:54:19.000169 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:19.000135 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Jul 02 13:54:19.000664 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:19.000643 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" event={"ID":"da6457ff-394d-4ad1-9f8f-a0e9e880dcce","Type":"ContainerStarted","Data":"0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe"} Jul 02 13:54:19.000840 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:19.000817 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:54:19.000945 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:19.000873 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:54:19.001767 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:19.001731 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Jul 02 13:54:19.002874 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:19.002851 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" event={"ID":"6d4dc493-0eab-44a2-b0f9-9cf16928015d","Type":"ContainerStarted","Data":"c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540"} Jul 02 13:54:19.002973 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:19.002964 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:54:19.033667 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:19.033618 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podStartSLOduration=2.721376652 podStartE2EDuration="38.03360314s" totalImagesPullingTime="31.728019236s" totalInitContainerRuntime="3.584207252s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:53:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:54:19.032197502 +0000 UTC m=+625.358758557" watchObservedRunningTime="2026-07-02 13:54:19.03360314 +0000 UTC m=+625.360164195" Jul 02 13:54:19.065729 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:19.065675 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podStartSLOduration=2.606682827 podStartE2EDuration="38.065660415s" totalImagesPullingTime="31.699371056s" totalInitContainerRuntime="3.759606532s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:53:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:54:19.064283445 +0000 UTC m=+625.390844555" watchObservedRunningTime="2026-07-02 13:54:19.065660415 +0000 UTC m=+625.392221470" Jul 02 13:54:19.104989 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:19.104939 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podStartSLOduration=2.809622079 podStartE2EDuration="37.104923855s" totalImagesPullingTime="30.713025992s" totalInitContainerRuntime="3.582275784s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:53:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:54:19.103432718 +0000 UTC m=+625.429993772" watchObservedRunningTime="2026-07-02 13:54:19.104923855 +0000 UTC m=+625.431484909" Jul 02 13:54:20.006895 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:20.006859 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Jul 02 13:54:20.007356 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:20.006868 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Jul 02 13:54:20.007356 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:20.007194 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:54:20.008171 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:20.008149 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 02 13:54:21.011530 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:21.011489 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 02 13:54:25.011367 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:25.011335 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:54:25.011879 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:25.011861 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:54:25.011977 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:25.011917 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Jul 02 13:54:25.012313 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:25.012288 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Jul 02 13:54:26.015863 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:26.015835 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:54:26.016381 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:26.016356 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 02 13:54:35.012858 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:35.012819 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Jul 02 13:54:35.013246 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:35.012817 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Jul 02 13:54:36.017053 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:36.016996 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 02 13:54:39.486896 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:39.486857 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:54:45.012281 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:45.012235 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Jul 02 13:54:45.012821 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:45.012366 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Jul 02 13:54:46.017055 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:46.016996 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 02 13:54:53.987758 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:53.987720 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:54:55.011978 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:55.011927 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Jul 02 13:54:55.012412 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:55.012385 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Jul 02 13:54:56.016866 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:54:56.016823 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 02 13:55:01.497497 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:01.497453 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:55:01.741502 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:01.741475 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:55:05.012495 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:05.012454 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Jul 02 13:55:05.012888 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:05.012454 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Jul 02 13:55:06.016474 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:06.016429 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 02 13:55:06.995297 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:06.995258 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:55:15.012372 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:15.012325 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Jul 02 13:55:15.012829 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:15.012326 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Jul 02 13:55:15.640860 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:15.640825 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:55:15.816655 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:15.816620 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:55:15.928111 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:15.928012 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:55:16.004173 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:16.004136 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:55:16.016672 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:16.016638 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 02 13:55:16.140586 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:16.140540 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:55:16.342842 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:16.342799 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:55:25.012226 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:25.012184 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Jul 02 13:55:25.012915 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:25.012897 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:55:26.016638 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:26.016603 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 02 13:55:35.012209 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:35.012177 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:55:36.017000 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:55:36.016972 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:56:01.584657 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:01.584562 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:01.896095 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:01.895974 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:11.749727 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:11.749689 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:11.838670 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:11.838634 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw"] Jul 02 13:56:11.838970 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:11.838922 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" containerID="cri-o://7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8" gracePeriod=30 Jul 02 13:56:11.839100 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:11.838980 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kube-rbac-proxy" containerID="cri-o://0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe" gracePeriod=30 Jul 02 13:56:11.871726 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:11.871689 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:11.907252 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:11.907219 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f"] Jul 02 13:56:11.907513 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:11.907484 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" containerID="cri-o://1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2" gracePeriod=30 Jul 02 13:56:11.907604 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:11.907519 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kube-rbac-proxy" containerID="cri-o://c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540" gracePeriod=30 Jul 02 13:56:11.977525 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:11.977489 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:12.051018 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.050980 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg"] Jul 02 13:56:12.051315 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.051287 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" containerID="cri-o://30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee" gracePeriod=30 Jul 02 13:56:12.051410 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.051369 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kube-rbac-proxy" containerID="cri-o://962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845" gracePeriod=30 Jul 02 13:56:12.059442 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.059415 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:12.134492 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.134438 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:12.234676 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.234638 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:12.498431 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.498393 2561 generic.go:356] "Generic (PLEG): container finished" podID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerID="0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe" exitCode=2 Jul 02 13:56:12.498600 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.498466 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" event={"ID":"da6457ff-394d-4ad1-9f8f-a0e9e880dcce","Type":"ContainerDied","Data":"0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe"} Jul 02 13:56:12.500556 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.500533 2561 generic.go:356] "Generic (PLEG): container finished" podID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerID="c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540" exitCode=2 Jul 02 13:56:12.500693 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.500603 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" event={"ID":"6d4dc493-0eab-44a2-b0f9-9cf16928015d","Type":"ContainerDied","Data":"c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540"} Jul 02 13:56:12.502615 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.502595 2561 generic.go:356] "Generic (PLEG): container finished" podID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerID="962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845" exitCode=2 Jul 02 13:56:12.502707 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.502627 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" event={"ID":"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b","Type":"ContainerDied","Data":"962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845"} Jul 02 13:56:12.691874 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:12.691837 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:15.007747 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.007699 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.29:8643/healthz\": dial tcp 10.132.0.29:8643: connect: connection refused" Jul 02 13:56:15.007747 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.007731 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused" Jul 02 13:56:15.012175 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.012147 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Jul 02 13:56:15.012365 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.012345 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Jul 02 13:56:15.798484 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.798452 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:56:15.904210 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.904121 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:15.926113 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.926082 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-proxy-tls\") pod \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " Jul 02 13:56:15.926304 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.926126 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm5m8\" (UniqueName: \"kubernetes.io/projected/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-api-access-wm5m8\") pod \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " Jul 02 13:56:15.926304 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.926174 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kserve-provision-location\") pod \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " Jul 02 13:56:15.926304 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.926231 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-rbac-proxy-sar-config\") pod \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " Jul 02 13:56:15.926464 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.926305 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-cabundle-cert\") pod \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\" (UID: \"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b\") " Jul 02 13:56:15.926527 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.926497 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" (UID: "d724e45d-5e3c-4ac9-bc2d-891d8da92e1b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:56:15.926594 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.926581 2561 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kserve-provision-location\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:15.926645 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.926616 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" (UID: "d724e45d-5e3c-4ac9-bc2d-891d8da92e1b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:56:15.926645 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.926623 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" (UID: "d724e45d-5e3c-4ac9-bc2d-891d8da92e1b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:56:15.928532 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.928506 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" (UID: "d724e45d-5e3c-4ac9-bc2d-891d8da92e1b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:56:15.928532 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:15.928514 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-api-access-wm5m8" (OuterVolumeSpecName: "kube-api-access-wm5m8") pod "d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" (UID: "d724e45d-5e3c-4ac9-bc2d-891d8da92e1b"). InnerVolumeSpecName "kube-api-access-wm5m8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:56:16.011704 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.011668 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.27:8643/healthz\": dial tcp 10.132.0.27:8643: connect: connection refused" Jul 02 13:56:16.016263 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.016235 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 02 13:56:16.027328 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.027299 2561 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-cabundle-cert\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:16.027446 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.027348 2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-proxy-tls\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:16.027446 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.027362 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm5m8\" (UniqueName: \"kubernetes.io/projected/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-api-access-wm5m8\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:16.027446 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.027375 2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:16.178811 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.178700 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:16.521139 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.521105 2561 generic.go:356] "Generic (PLEG): container finished" podID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerID="30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee" exitCode=0 Jul 02 13:56:16.521331 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.521180 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" Jul 02 13:56:16.521331 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.521190 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" event={"ID":"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b","Type":"ContainerDied","Data":"30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee"} Jul 02 13:56:16.521331 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.521253 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg" event={"ID":"d724e45d-5e3c-4ac9-bc2d-891d8da92e1b","Type":"ContainerDied","Data":"a1d4fdcf2a08d6f159fc9f6350e72e942dce685634c84153ffc3a1750f644369"} Jul 02 13:56:16.521331 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.521291 2561 scope.go:117] "RemoveContainer" containerID="962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845" Jul 02 13:56:16.529753 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.529732 2561 scope.go:117] "RemoveContainer" containerID="30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee" Jul 02 13:56:16.541544 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.541515 2561 scope.go:117] "RemoveContainer" containerID="cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d" Jul 02 13:56:16.543002 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.542982 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg"] Jul 02 13:56:16.547988 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.547965 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-72pkg"] Jul 02 13:56:16.550073 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.550058 2561 scope.go:117] "RemoveContainer" containerID="962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845" Jul 02 13:56:16.550327 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:56:16.550310 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845\": container with ID starting with 962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845 not found: ID does not exist" containerID="962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845" Jul 02 13:56:16.550379 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.550337 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845"} err="failed to get container status \"962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845\": rpc error: code = NotFound desc = could not find container \"962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845\": container with ID starting with 962fa523f656b6bc1689cbf9a25f3f92f75a369658d30d826f714113c5a6c845 not found: ID does not exist" Jul 02 13:56:16.550379 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.550354 2561 scope.go:117] "RemoveContainer" containerID="30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee" Jul 02 13:56:16.550653 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:56:16.550625 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee\": container with ID starting with 30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee not found: ID does not exist" containerID="30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee" Jul 02 13:56:16.550714 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.550657 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee"} err="failed to get container status \"30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee\": rpc error: code = NotFound desc = could not find container \"30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee\": container with ID starting with 30e97caf6bb3c16f33b3f405a5573bc6689a1b330c9a19709c9658e67d1be0ee not found: ID does not exist" Jul 02 13:56:16.550714 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.550674 2561 scope.go:117] "RemoveContainer" containerID="cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d" Jul 02 13:56:16.550907 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:56:16.550884 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d\": container with ID starting with cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d not found: ID does not exist" containerID="cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d" Jul 02 13:56:16.550955 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:16.550905 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d"} err="failed to get container status \"cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d\": rpc error: code = NotFound desc = could not find container \"cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d\": container with ID starting with cac2ee8e706861a8d7c39f884207da548497669d66e1144cbf8ee52c96537a5d not found: ID does not exist" Jul 02 13:56:17.067232 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.067206 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:56:17.091500 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.091402 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:56:17.138252 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138221 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-proxy-tls\") pod \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " Jul 02 13:56:17.138445 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138259 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-cabundle-cert\") pod \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " Jul 02 13:56:17.138445 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138281 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-cabundle-cert\") pod \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " Jul 02 13:56:17.138445 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138319 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-rbac-proxy-sar-config\") pod \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " Jul 02 13:56:17.138445 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138363 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d4dc493-0eab-44a2-b0f9-9cf16928015d-proxy-tls\") pod \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " Jul 02 13:56:17.138445 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138380 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kserve-provision-location\") pod \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " Jul 02 13:56:17.138445 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138402 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq88q\" (UniqueName: \"kubernetes.io/projected/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-api-access-xq88q\") pod \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " Jul 02 13:56:17.138445 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138443 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-rbac-proxy-sar-config\") pod \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\" (UID: \"6d4dc493-0eab-44a2-b0f9-9cf16928015d\") " Jul 02 13:56:17.138803 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138497 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kserve-provision-location\") pod \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " Jul 02 13:56:17.138803 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138548 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9792\" (UniqueName: \"kubernetes.io/projected/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-api-access-z9792\") pod \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\" (UID: \"da6457ff-394d-4ad1-9f8f-a0e9e880dcce\") " Jul 02 13:56:17.138803 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138674 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "6d4dc493-0eab-44a2-b0f9-9cf16928015d" (UID: "6d4dc493-0eab-44a2-b0f9-9cf16928015d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:56:17.138803 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138676 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "da6457ff-394d-4ad1-9f8f-a0e9e880dcce" (UID: "da6457ff-394d-4ad1-9f8f-a0e9e880dcce"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:56:17.138803 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.138729 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "da6457ff-394d-4ad1-9f8f-a0e9e880dcce" (UID: "da6457ff-394d-4ad1-9f8f-a0e9e880dcce"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:56:17.139076 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.139001 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "da6457ff-394d-4ad1-9f8f-a0e9e880dcce" (UID: "da6457ff-394d-4ad1-9f8f-a0e9e880dcce"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:56:17.139076 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.139010 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "6d4dc493-0eab-44a2-b0f9-9cf16928015d" (UID: "6d4dc493-0eab-44a2-b0f9-9cf16928015d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:56:17.139188 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.139135 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6d4dc493-0eab-44a2-b0f9-9cf16928015d" (UID: "6d4dc493-0eab-44a2-b0f9-9cf16928015d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:56:17.139246 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.139189 2561 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kserve-provision-location\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:17.139246 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.139209 2561 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-cabundle-cert\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:17.139246 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.139224 2561 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-cabundle-cert\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:17.139246 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.139238 2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:17.139455 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.139252 2561 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kserve-provision-location\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:17.140777 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.140749 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "da6457ff-394d-4ad1-9f8f-a0e9e880dcce" (UID: "da6457ff-394d-4ad1-9f8f-a0e9e880dcce"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:56:17.141361 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.141337 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-api-access-z9792" (OuterVolumeSpecName: "kube-api-access-z9792") pod "da6457ff-394d-4ad1-9f8f-a0e9e880dcce" (UID: "da6457ff-394d-4ad1-9f8f-a0e9e880dcce"). InnerVolumeSpecName "kube-api-access-z9792". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:56:17.141474 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.141455 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d4dc493-0eab-44a2-b0f9-9cf16928015d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6d4dc493-0eab-44a2-b0f9-9cf16928015d" (UID: "6d4dc493-0eab-44a2-b0f9-9cf16928015d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:56:17.141671 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.141653 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-api-access-xq88q" (OuterVolumeSpecName: "kube-api-access-xq88q") pod "6d4dc493-0eab-44a2-b0f9-9cf16928015d" (UID: "6d4dc493-0eab-44a2-b0f9-9cf16928015d"). InnerVolumeSpecName "kube-api-access-xq88q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:56:17.240125 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.240088 2561 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:17.240125 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.240119 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9792\" (UniqueName: \"kubernetes.io/projected/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-kube-api-access-z9792\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:17.240125 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.240131 2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da6457ff-394d-4ad1-9f8f-a0e9e880dcce-proxy-tls\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:17.240379 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.240146 2561 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d4dc493-0eab-44a2-b0f9-9cf16928015d-proxy-tls\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:17.240379 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.240154 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq88q\" (UniqueName: \"kubernetes.io/projected/6d4dc493-0eab-44a2-b0f9-9cf16928015d-kube-api-access-xq88q\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 13:56:17.527242 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.527205 2561 generic.go:356] "Generic (PLEG): container finished" podID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerID="7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8" exitCode=0 Jul 02 13:56:17.527433 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.527267 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" Jul 02 13:56:17.527433 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.527289 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" event={"ID":"da6457ff-394d-4ad1-9f8f-a0e9e880dcce","Type":"ContainerDied","Data":"7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8"} Jul 02 13:56:17.527433 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.527375 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw" event={"ID":"da6457ff-394d-4ad1-9f8f-a0e9e880dcce","Type":"ContainerDied","Data":"86da81173ff481593846ec1cba6cd78434e20ed4b263d9c190711b6e632ea923"} Jul 02 13:56:17.527433 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.527417 2561 scope.go:117] "RemoveContainer" containerID="0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe" Jul 02 13:56:17.529302 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.529278 2561 generic.go:356] "Generic (PLEG): container finished" podID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerID="1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2" exitCode=0 Jul 02 13:56:17.529422 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.529325 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" Jul 02 13:56:17.529422 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.529356 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" event={"ID":"6d4dc493-0eab-44a2-b0f9-9cf16928015d","Type":"ContainerDied","Data":"1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2"} Jul 02 13:56:17.529422 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.529417 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f" event={"ID":"6d4dc493-0eab-44a2-b0f9-9cf16928015d","Type":"ContainerDied","Data":"c58198e93ccb42bad1832da38a1f0f662e3be1560e5d2352181d3f4c851270dc"} Jul 02 13:56:17.537308 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.537157 2561 scope.go:117] "RemoveContainer" containerID="7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8" Jul 02 13:56:17.548713 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.548691 2561 scope.go:117] "RemoveContainer" containerID="a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34" Jul 02 13:56:17.556485 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.556470 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f"] Jul 02 13:56:17.556560 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.556514 2561 scope.go:117] "RemoveContainer" containerID="0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe" Jul 02 13:56:17.556795 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:56:17.556776 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe\": container with ID starting with 0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe not found: ID does not exist" containerID="0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe" Jul 02 13:56:17.556871 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.556806 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe"} err="failed to get container status \"0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe\": rpc error: code = NotFound desc = could not find container \"0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe\": container with ID starting with 0b3f4c23f856eda2d086c2865fdc25a3dc2c5e4bc4b61b85d3fa62985269d3fe not found: ID does not exist" Jul 02 13:56:17.556871 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.556831 2561 scope.go:117] "RemoveContainer" containerID="7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8" Jul 02 13:56:17.557057 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:56:17.557028 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8\": container with ID starting with 7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8 not found: ID does not exist" containerID="7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8" Jul 02 13:56:17.557118 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.557062 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8"} err="failed to get container status \"7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8\": rpc error: code = NotFound desc = could not find container \"7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8\": container with ID starting with 7ac3314f71dd67c452b441252aa782a578991410ec7c9268b846fcba6390ded8 not found: ID does not exist" Jul 02 13:56:17.557118 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.557078 2561 scope.go:117] "RemoveContainer" containerID="a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34" Jul 02 13:56:17.557314 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:56:17.557297 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34\": container with ID starting with a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34 not found: ID does not exist" containerID="a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34" Jul 02 13:56:17.557355 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.557320 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34"} err="failed to get container status \"a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34\": rpc error: code = NotFound desc = could not find container \"a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34\": container with ID starting with a5b6e01989e55807147178983d05a3900aa288130d76288f5e61a51c9a6e9c34 not found: ID does not exist" Jul 02 13:56:17.557355 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.557334 2561 scope.go:117] "RemoveContainer" containerID="c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540" Jul 02 13:56:17.562234 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.562213 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-866d764d98-72g5f"] Jul 02 13:56:17.564845 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.564818 2561 scope.go:117] "RemoveContainer" containerID="1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2" Jul 02 13:56:17.572074 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.572057 2561 scope.go:117] "RemoveContainer" containerID="be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea" Jul 02 13:56:17.573453 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.573434 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw"] Jul 02 13:56:17.578704 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.578684 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-555c647c7d-2z4lw"] Jul 02 13:56:17.579438 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.579426 2561 scope.go:117] "RemoveContainer" containerID="c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540" Jul 02 13:56:17.579681 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:56:17.579663 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540\": container with ID starting with c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540 not found: ID does not exist" containerID="c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540" Jul 02 13:56:17.579722 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.579688 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540"} err="failed to get container status \"c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540\": rpc error: code = NotFound desc = could not find container \"c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540\": container with ID starting with c3e495e605c1947b2f1ff6df958cd006e9d46441e6a5a0ac9011d45af8878540 not found: ID does not exist" Jul 02 13:56:17.579722 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.579706 2561 scope.go:117] "RemoveContainer" containerID="1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2" Jul 02 13:56:17.579955 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:56:17.579939 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2\": container with ID starting with 1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2 not found: ID does not exist" containerID="1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2" Jul 02 13:56:17.580001 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.579958 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2"} err="failed to get container status \"1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2\": rpc error: code = NotFound desc = could not find container \"1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2\": container with ID starting with 1faed0e9c75dce7f596a8406c9cccb9c9a92c923c2e5950bc2e8e35be14f74a2 not found: ID does not exist" Jul 02 13:56:17.580001 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.579971 2561 scope.go:117] "RemoveContainer" containerID="be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea" Jul 02 13:56:17.580194 ip-10-0-137-40 kubenswrapper[2561]: E0702 13:56:17.580168 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea\": container with ID starting with be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea not found: ID does not exist" containerID="be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea" Jul 02 13:56:17.580194 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:17.580186 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea"} err="failed to get container status \"be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea\": rpc error: code = NotFound desc = could not find container \"be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea\": container with ID starting with be96babb266ddac278a53d2636f5001a9e509a68c0e5e2eab8b6777e72f111ea not found: ID does not exist" Jul 02 13:56:18.086431 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:18.086393 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" path="/var/lib/kubelet/pods/6d4dc493-0eab-44a2-b0f9-9cf16928015d/volumes" Jul 02 13:56:18.086905 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:18.086891 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" path="/var/lib/kubelet/pods/d724e45d-5e3c-4ac9-bc2d-891d8da92e1b/volumes" Jul 02 13:56:18.087631 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:18.087615 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" path="/var/lib/kubelet/pods/da6457ff-394d-4ad1-9f8f-a0e9e880dcce/volumes" Jul 02 13:56:32.989747 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:32.989707 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:56:33.049648 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:56:33.049618 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:57:11.937759 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:57:11.937683 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:57:12.248980 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:57:12.248945 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 13:59:15.433488 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:59:15.433453 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 13:59:15.435889 ip-10-0-137-40 kubenswrapper[2561]: I0702 13:59:15.434976 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:00:43.528225 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:00:43.528190 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:03:02.572431 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:03:02.572391 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:04:15.461373 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:04:15.461261 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:04:15.465559 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:04:15.463594 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:04:26.998220 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:04:26.998186 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:04:30.422633 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:04:30.422597 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:04:30.548487 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:04:30.548444 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:04:30.630871 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:04:30.630835 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:04:30.719765 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:04:30.719669 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:04:30.777795 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:04:30.777742 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:04:30.892937 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:04:30.892898 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:04:31.031390 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:04:31.031359 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:26.637377 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:26.637342 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:26.735496 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:26.735445 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:26.835600 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:26.835565 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:26.910497 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:26.910411 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:27.069498 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:27.069467 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:27.220760 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:27.220669 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:30.630833 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:30.630804 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:30.912635 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:30.912527 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:40.732706 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:40.732673 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:40.813015 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:40.812973 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:40.886858 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:40.886825 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:40.963173 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:40.963142 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:41.218408 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:41.218317 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:05:41.318906 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:05:41.318865 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:06:26.808861 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:06:26.808763 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:06:28.462503 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:06:28.462463 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:06:36.925334 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:06:36.925302 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:06:37.105318 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:06:37.105272 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:06:37.205237 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:06:37.205137 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:06:37.305382 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:06:37.305344 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:06:37.428489 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:06:37.428432 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:06:40.924689 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:06:40.924657 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:06:41.300482 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:06:41.300448 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:07:37.112690 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:07:37.112654 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:07:37.217863 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:07:37.217823 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:07:47.680014 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:07:47.679922 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:08:48.524602 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:08:48.524560 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:08:58.704269 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:08:58.704222 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:09:15.493668 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:09:15.493640 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:09:15.495919 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:09:15.494690 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:11:10.022919 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:11:10.022832 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:13:12.553247 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:13:12.553208 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:14:00.088656 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:14:00.088620 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:14:15.520554 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:14:15.520428 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:14:15.529111 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:14:15.521282 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:14:55.693639 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:14:55.693604 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:14:55.836737 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:14:55.836696 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:14:55.926260 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:14:55.926230 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:14:56.015895 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:14:56.015860 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:14:56.116654 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:14:56.116614 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:14:56.397611 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:14:56.397521 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:15:51.708590 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:15:51.708550 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:15:51.850540 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:15:51.850504 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:15:51.952355 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:15:51.952317 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:15:52.060421 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:15:52.060388 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:15:52.139273 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:15:52.139239 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:15:52.394837 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:15:52.394746 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:15:55.992485 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:15:55.992454 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:15:56.154227 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:15:56.154179 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:16:06.016484 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:16:06.016434 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:16:06.147154 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:16:06.147096 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:16:06.243594 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:16:06.243564 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:16:06.895118 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:16:06.895085 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:16:06.998520 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:16:06.998484 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:16:51.959808 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:16:51.959724 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:16:52.958700 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:16:52.958667 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:06.226376 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:17:06.226345 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:06.708265 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:17:06.708230 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:59.073567 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:18:59.073532 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:01.990435 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:19:01.990402 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:15.550402 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:19:15.550301 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:19:15.554426 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:19:15.551981 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:20:22.829125 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:20:22.829028 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:22:21.203679 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:22:21.203648 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:24:03.570728 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:24:03.570690 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:24:15.576775 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:24:15.576660 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:24:15.580703 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:24:15.578992 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:24:24.515491 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:24:24.515414 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:20.855297 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:25:20.855261 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:20.985557 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:25:20.985517 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:21.081492 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:25:21.081458 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:27:27.287405 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:27:27.287323 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:29:15.602600 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:29:15.602473 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:29:15.606617 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:29:15.605683 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:31:32.102385 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:31:32.102344 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:11.490593 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:11.490557 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:11.665229 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:11.665190 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:11.829536 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:11.829440 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:12.583691 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.583655 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-szblp/must-gather-9z28t"] Jul 02 14:33:12.584108 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584084 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="storage-initializer" Jul 02 14:33:12.584108 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584098 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="storage-initializer" Jul 02 14:33:12.584108 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584110 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="storage-initializer" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584116 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="storage-initializer" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584125 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kube-rbac-proxy" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584130 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kube-rbac-proxy" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584139 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kube-rbac-proxy" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584144 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kube-rbac-proxy" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584152 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="storage-initializer" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584157 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="storage-initializer" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584171 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kube-rbac-proxy" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584176 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kube-rbac-proxy" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584184 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584190 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584196 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584202 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584207 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" Jul 02 14:33:12.584238 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584212 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" Jul 02 14:33:12.584730 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584260 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kserve-container" Jul 02 14:33:12.584730 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584268 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kserve-container" Jul 02 14:33:12.584730 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584276 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kube-rbac-proxy" Jul 02 14:33:12.584730 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584282 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="da6457ff-394d-4ad1-9f8f-a0e9e880dcce" containerName="kserve-container" Jul 02 14:33:12.584730 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584288 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d724e45d-5e3c-4ac9-bc2d-891d8da92e1b" containerName="kube-rbac-proxy" Jul 02 14:33:12.584730 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.584294 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d4dc493-0eab-44a2-b0f9-9cf16928015d" containerName="kube-rbac-proxy" Jul 02 14:33:12.587506 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.587490 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:12.592361 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.592341 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-szblp\"/\"kube-root-ca.crt\"" Jul 02 14:33:12.592553 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.592535 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-szblp\"/\"openshift-service-ca.crt\"" Jul 02 14:33:12.593378 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.593360 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-szblp\"/\"default-dockercfg-4stf8\"" Jul 02 14:33:12.600833 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.600814 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-szblp/must-gather-9z28t"] Jul 02 14:33:12.601285 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.601265 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:12.691334 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.691294 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpkv7\" (UniqueName: \"kubernetes.io/projected/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-kube-api-access-lpkv7\") pod \"must-gather-9z28t\" (UID: \"2d2a7faf-e66e-43cf-8d82-4b3d1087452d\") " pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:12.691489 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.691340 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-must-gather-output\") pod \"must-gather-9z28t\" (UID: \"2d2a7faf-e66e-43cf-8d82-4b3d1087452d\") " pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:12.702718 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.702689 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:12.791876 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.791836 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpkv7\" (UniqueName: \"kubernetes.io/projected/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-kube-api-access-lpkv7\") pod \"must-gather-9z28t\" (UID: \"2d2a7faf-e66e-43cf-8d82-4b3d1087452d\") " pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:12.791876 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.791877 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-must-gather-output\") pod \"must-gather-9z28t\" (UID: \"2d2a7faf-e66e-43cf-8d82-4b3d1087452d\") " pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:12.792282 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.792265 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-must-gather-output\") pod \"must-gather-9z28t\" (UID: \"2d2a7faf-e66e-43cf-8d82-4b3d1087452d\") " pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:12.805409 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.805381 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpkv7\" (UniqueName: \"kubernetes.io/projected/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-kube-api-access-lpkv7\") pod \"must-gather-9z28t\" (UID: \"2d2a7faf-e66e-43cf-8d82-4b3d1087452d\") " pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:12.912790 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:12.912683 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:13.049142 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:13.049112 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-szblp/must-gather-9z28t"] Jul 02 14:33:13.051754 ip-10-0-137-40 kubenswrapper[2561]: W0702 14:33:13.051721 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d2a7faf_e66e_43cf_8d82_4b3d1087452d.slice/crio-aede5b00ff4369cc98398ebe33b195cc1e0128d8c0fd5856caf72cd6f6285784 WatchSource:0}: Error finding container aede5b00ff4369cc98398ebe33b195cc1e0128d8c0fd5856caf72cd6f6285784: Status 404 returned error can't find the container with id aede5b00ff4369cc98398ebe33b195cc1e0128d8c0fd5856caf72cd6f6285784 Jul 02 14:33:13.053375 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:13.053359 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 14:33:13.336383 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:13.336347 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-szblp/must-gather-9z28t" event={"ID":"2d2a7faf-e66e-43cf-8d82-4b3d1087452d","Type":"ContainerStarted","Data":"aede5b00ff4369cc98398ebe33b195cc1e0128d8c0fd5856caf72cd6f6285784"} Jul 02 14:33:19.374621 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:19.374588 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-szblp/must-gather-9z28t" event={"ID":"2d2a7faf-e66e-43cf-8d82-4b3d1087452d","Type":"ContainerStarted","Data":"3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c"} Jul 02 14:33:20.381739 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:20.381704 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-szblp/must-gather-9z28t" event={"ID":"2d2a7faf-e66e-43cf-8d82-4b3d1087452d","Type":"ContainerStarted","Data":"dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba"} Jul 02 14:33:20.403658 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:20.403588 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-szblp/must-gather-9z28t" podStartSLOduration=2.316985288 podStartE2EDuration="8.403564804s" totalImagesPullingTime="6.086579516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:33:20.401086957 +0000 UTC m=+2966.727648007" watchObservedRunningTime="2026-07-02 14:33:20.403564804 +0000 UTC m=+2966.730125860" Jul 02 14:33:43.484480 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:43.484444 2561 generic.go:356] "Generic (PLEG): container finished" podID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" containerID="3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c" exitCode=0 Jul 02 14:33:43.484896 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:43.484492 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-szblp/must-gather-9z28t" event={"ID":"2d2a7faf-e66e-43cf-8d82-4b3d1087452d","Type":"ContainerDied","Data":"3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c"} Jul 02 14:33:43.484896 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:43.484770 2561 scope.go:117] "RemoveContainer" containerID="3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c" Jul 02 14:33:43.524860 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:43.524831 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-szblp_must-gather-9z28t_2d2a7faf-e66e-43cf-8d82-4b3d1087452d/gather/0.log" Jul 02 14:33:44.015855 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:44.015817 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:44.199183 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:44.199147 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:44.298524 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:44.298400 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:47.305568 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:47.305512 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bj8sb_c0b8e79b-9ee6-4442-bb4b-5020ff3833a9/global-pull-secret-syncer/0.log" Jul 02 14:33:47.394880 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:47.394845 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" Jul 02 14:33:47.518480 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:47.518447 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-40.ec2.internal_c02ef9b20a4c8c441a824f8fcd9226be/haproxy/0.log" Jul 02 14:33:48.995496 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:48.995464 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-szblp/must-gather-9z28t"] Jul 02 14:33:48.995977 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:48.995649 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-szblp/must-gather-9z28t" podUID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" containerName="copy" containerID="cri-o://dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba" gracePeriod=2 Jul 02 14:33:49.002108 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.002082 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-szblp/must-gather-9z28t"] Jul 02 14:33:49.003529 ip-10-0-137-40 kubenswrapper[2561]: E0702 14:33:49.003501 2561 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"2d2a7faf-e66e-43cf-8d82-4b3d1087452d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"openshift-must-gather-szblp\"/\"must-gather-9z28t\": pods \"must-gather-9z28t\" not found" pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:49.227783 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.227758 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-szblp_must-gather-9z28t_2d2a7faf-e66e-43cf-8d82-4b3d1087452d/copy/0.log" Jul 02 14:33:49.229563 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.229546 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:49.232026 ip-10-0-137-40 kubenswrapper[2561]: E0702 14:33:49.231999 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-9z28t\" is forbidden: User \"system:node:ip-10-0-137-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-szblp\": no relationship found between node 'ip-10-0-137-40.ec2.internal' and this object" podUID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:49.313202 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.313111 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-must-gather-output\") pod \"2d2a7faf-e66e-43cf-8d82-4b3d1087452d\" (UID: \"2d2a7faf-e66e-43cf-8d82-4b3d1087452d\") " Jul 02 14:33:49.313202 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.313196 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpkv7\" (UniqueName: \"kubernetes.io/projected/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-kube-api-access-lpkv7\") pod \"2d2a7faf-e66e-43cf-8d82-4b3d1087452d\" (UID: \"2d2a7faf-e66e-43cf-8d82-4b3d1087452d\") " Jul 02 14:33:49.314609 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.314578 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "2d2a7faf-e66e-43cf-8d82-4b3d1087452d" (UID: "2d2a7faf-e66e-43cf-8d82-4b3d1087452d"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:33:49.315397 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.315369 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-kube-api-access-lpkv7" (OuterVolumeSpecName: "kube-api-access-lpkv7") pod "2d2a7faf-e66e-43cf-8d82-4b3d1087452d" (UID: "2d2a7faf-e66e-43cf-8d82-4b3d1087452d"). InnerVolumeSpecName "kube-api-access-lpkv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:33:49.414441 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.414405 2561 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-must-gather-output\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 14:33:49.414441 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.414432 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpkv7\" (UniqueName: \"kubernetes.io/projected/2d2a7faf-e66e-43cf-8d82-4b3d1087452d-kube-api-access-lpkv7\") on node \"ip-10-0-137-40.ec2.internal\" DevicePath \"\"" Jul 02 14:33:49.509619 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.509592 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-szblp_must-gather-9z28t_2d2a7faf-e66e-43cf-8d82-4b3d1087452d/copy/0.log" Jul 02 14:33:49.509973 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.509949 2561 generic.go:356] "Generic (PLEG): container finished" podID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" containerID="dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba" exitCode=143 Jul 02 14:33:49.510094 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.510001 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:49.510094 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.510060 2561 scope.go:117] "RemoveContainer" containerID="dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba" Jul 02 14:33:49.512533 ip-10-0-137-40 kubenswrapper[2561]: E0702 14:33:49.512498 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-9z28t\" is forbidden: User \"system:node:ip-10-0-137-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-szblp\": no relationship found between node 'ip-10-0-137-40.ec2.internal' and this object" podUID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:49.518956 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.518940 2561 scope.go:117] "RemoveContainer" containerID="3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c" Jul 02 14:33:49.520493 ip-10-0-137-40 kubenswrapper[2561]: E0702 14:33:49.520469 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-9z28t\" is forbidden: User \"system:node:ip-10-0-137-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-szblp\": no relationship found between node 'ip-10-0-137-40.ec2.internal' and this object" podUID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" pod="openshift-must-gather-szblp/must-gather-9z28t" Jul 02 14:33:49.530898 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.530879 2561 scope.go:117] "RemoveContainer" containerID="dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba" Jul 02 14:33:49.531156 ip-10-0-137-40 kubenswrapper[2561]: E0702 14:33:49.531139 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba\": container with ID starting with dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba not found: ID does not exist" containerID="dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba" Jul 02 14:33:49.531225 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.531162 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba"} err="failed to get container status \"dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba\": rpc error: code = NotFound desc = could not find container \"dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba\": container with ID starting with dd5e7c5246a93ff6ae9719e95d0964b1276bcd1596b4125c86d8cd1dc4416cba not found: ID does not exist" Jul 02 14:33:49.531225 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.531178 2561 scope.go:117] "RemoveContainer" containerID="3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c" Jul 02 14:33:49.531413 ip-10-0-137-40 kubenswrapper[2561]: E0702 14:33:49.531399 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c\": container with ID starting with 3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c not found: ID does not exist" containerID="3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c" Jul 02 14:33:49.531458 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.531416 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c"} err="failed to get container status \"3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c\": rpc error: code = NotFound desc = could not find container \"3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c\": container with ID starting with 3cc08db15618ef4ba2db5dd4b8be0af1014883b11b654f8872400c02f3710e6c not found: ID does not exist" Jul 02 14:33:49.790285 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.790228 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:49.856743 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:49.856710 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:50.086521 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:50.086435 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" path="/var/lib/kubelet/pods/2d2a7faf-e66e-43cf-8d82-4b3d1087452d/volumes" Jul 02 14:33:51.184886 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.184857 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_29359fc4-11ce-4837-bedf-96f78dfe18e0/alertmanager/0.log" Jul 02 14:33:51.219236 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.219207 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_29359fc4-11ce-4837-bedf-96f78dfe18e0/config-reloader/0.log" Jul 02 14:33:51.256274 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.256247 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_29359fc4-11ce-4837-bedf-96f78dfe18e0/kube-rbac-proxy-web/0.log" Jul 02 14:33:51.288947 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.288917 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_29359fc4-11ce-4837-bedf-96f78dfe18e0/kube-rbac-proxy/0.log" Jul 02 14:33:51.327023 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.326961 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_29359fc4-11ce-4837-bedf-96f78dfe18e0/kube-rbac-proxy-metric/0.log" Jul 02 14:33:51.362182 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.362149 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_29359fc4-11ce-4837-bedf-96f78dfe18e0/prom-label-proxy/0.log" Jul 02 14:33:51.403845 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.403816 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_29359fc4-11ce-4837-bedf-96f78dfe18e0/init-config-reloader/0.log" Jul 02 14:33:51.454132 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.454062 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-6zctb_25a79a7e-2692-47b0-a51c-3a834dc4e8ff/cluster-monitoring-operator/0.log" Jul 02 14:33:51.495576 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.495549 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-c5fsr_0aff0d31-ef82-40c6-9108-6c665f163422/kube-state-metrics/0.log" Jul 02 14:33:51.526899 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.526875 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-c5fsr_0aff0d31-ef82-40c6-9108-6c665f163422/kube-rbac-proxy-main/0.log" Jul 02 14:33:51.563718 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.563691 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-c5fsr_0aff0d31-ef82-40c6-9108-6c665f163422/kube-rbac-proxy-self/0.log" Jul 02 14:33:51.605113 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.605081 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-85dd74cbb5-49dmh_bb03ddb5-82f7-4334-a5c4-749de4866b3e/metrics-server/0.log" Jul 02 14:33:51.802842 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.802816 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qbs4t_0f58a897-d774-4145-a977-1cfee9e900f9/node-exporter/0.log" Jul 02 14:33:51.832080 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.832031 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qbs4t_0f58a897-d774-4145-a977-1cfee9e900f9/kube-rbac-proxy/0.log" Jul 02 14:33:51.859583 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.859555 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qbs4t_0f58a897-d774-4145-a977-1cfee9e900f9/init-textfile/0.log" Jul 02 14:33:51.988854 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:51.988822 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-s5946_28eb750e-4066-4624-9014-58b0513e0cae/kube-rbac-proxy-main/0.log" Jul 02 14:33:52.046468 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:52.046440 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-s5946_28eb750e-4066-4624-9014-58b0513e0cae/kube-rbac-proxy-self/0.log" Jul 02 14:33:52.076850 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:52.076782 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-s5946_28eb750e-4066-4624-9014-58b0513e0cae/openshift-state-metrics/0.log" Jul 02 14:33:52.131854 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:52.131822 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_420157d1-d7e5-401e-a1a8-adf21ae0465d/prometheus/0.log" Jul 02 14:33:52.158727 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:52.158701 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_420157d1-d7e5-401e-a1a8-adf21ae0465d/config-reloader/0.log" Jul 02 14:33:52.186134 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:52.186110 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_420157d1-d7e5-401e-a1a8-adf21ae0465d/thanos-sidecar/0.log" Jul 02 14:33:52.214897 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:52.214873 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_420157d1-d7e5-401e-a1a8-adf21ae0465d/kube-rbac-proxy-web/0.log" Jul 02 14:33:52.251438 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:52.251406 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_420157d1-d7e5-401e-a1a8-adf21ae0465d/kube-rbac-proxy/0.log" Jul 02 14:33:52.279068 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:52.279034 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_420157d1-d7e5-401e-a1a8-adf21ae0465d/kube-rbac-proxy-thanos/0.log" Jul 02 14:33:52.309857 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:52.309830 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_420157d1-d7e5-401e-a1a8-adf21ae0465d/init-config-reloader/0.log" Jul 02 14:33:52.408600 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:52.408532 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-2n6c5_6422729e-0ce4-4015-8e95-d0bab0a02616/prometheus-operator-admission-webhook/0.log" Jul 02 14:33:54.993644 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:54.993610 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5499b6fb8f-mjsl9_8d0f83bd-b6c3-4e4c-8051-2b44721b2d29/download-server/0.log" Jul 02 14:33:55.886634 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.886597 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh"] Jul 02 14:33:55.886944 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.886932 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" containerName="gather" Jul 02 14:33:55.887005 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.886945 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" containerName="gather" Jul 02 14:33:55.887005 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.886963 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" containerName="copy" Jul 02 14:33:55.887005 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.886969 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" containerName="copy" Jul 02 14:33:55.887134 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.887017 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" containerName="gather" Jul 02 14:33:55.887134 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.887028 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d2a7faf-e66e-43cf-8d82-4b3d1087452d" containerName="copy" Jul 02 14:33:55.890550 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.890531 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:55.892996 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.892965 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p8hbs\"/\"openshift-service-ca.crt\"" Jul 02 14:33:55.892996 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.892965 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-p8hbs\"/\"default-dockercfg-4f72d\"" Jul 02 14:33:55.893159 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.893096 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p8hbs\"/\"kube-root-ca.crt\"" Jul 02 14:33:55.900848 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.900826 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh"] Jul 02 14:33:55.970095 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.970034 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-proc\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:55.970250 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.970169 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zw96\" (UniqueName: \"kubernetes.io/projected/404f5604-bee3-4025-a368-1e5535f79438-kube-api-access-6zw96\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:55.970250 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.970219 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-podres\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:55.970250 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.970243 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-sys\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:55.970395 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:55.970259 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-lib-modules\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.071281 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.071234 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zw96\" (UniqueName: \"kubernetes.io/projected/404f5604-bee3-4025-a368-1e5535f79438-kube-api-access-6zw96\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.071281 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.071290 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-podres\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.071712 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.071315 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-sys\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.071712 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.071330 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-lib-modules\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.071712 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.071373 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-proc\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.071712 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.071446 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-sys\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.071712 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.071464 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-proc\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.071712 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.071494 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-podres\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.071712 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.071516 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/404f5604-bee3-4025-a368-1e5535f79438-lib-modules\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.079889 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.079861 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zw96\" (UniqueName: \"kubernetes.io/projected/404f5604-bee3-4025-a368-1e5535f79438-kube-api-access-6zw96\") pod \"perf-node-gather-daemonset-kqwrh\" (UID: \"404f5604-bee3-4025-a368-1e5535f79438\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.201983 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.201896 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.330554 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.330530 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh"] Jul 02 14:33:56.333091 ip-10-0-137-40 kubenswrapper[2561]: W0702 14:33:56.333063 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod404f5604_bee3_4025_a368_1e5535f79438.slice/crio-c22a1406e5c436e3f7e33b1075dfdc72c8606b7977cc41ebd3e7ff1f02e90959 WatchSource:0}: Error finding container c22a1406e5c436e3f7e33b1075dfdc72c8606b7977cc41ebd3e7ff1f02e90959: Status 404 returned error can't find the container with id c22a1406e5c436e3f7e33b1075dfdc72c8606b7977cc41ebd3e7ff1f02e90959 Jul 02 14:33:56.540419 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.540379 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" event={"ID":"404f5604-bee3-4025-a368-1e5535f79438","Type":"ContainerStarted","Data":"3713d3c6829229984199b0db8bad88d70c6fa5d42c3a788fcde28affe5c3e473"} Jul 02 14:33:56.540597 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.540438 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" event={"ID":"404f5604-bee3-4025-a368-1e5535f79438","Type":"ContainerStarted","Data":"c22a1406e5c436e3f7e33b1075dfdc72c8606b7977cc41ebd3e7ff1f02e90959"} Jul 02 14:33:56.540597 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.540496 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:33:56.562906 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.562849 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" podStartSLOduration=1.562834026 podStartE2EDuration="1.562834026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:33:56.561378823 +0000 UTC m=+3002.887939879" watchObservedRunningTime="2026-07-02 14:33:56.562834026 +0000 UTC m=+3002.889395081" Jul 02 14:33:56.658060 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.658023 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pbgrb_82c5a9c0-96fc-4f4b-8194-83a58934ba8f/dns/0.log" Jul 02 14:33:56.694330 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.694301 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pbgrb_82c5a9c0-96fc-4f4b-8194-83a58934ba8f/kube-rbac-proxy/0.log" Jul 02 14:33:56.806173 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:56.806093 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9tw66_85ec7384-3925-4a95-a743-fd26e061cb25/dns-node-resolver/0.log" Jul 02 14:33:57.557417 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:57.557386 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-slrnm_a4e02fa4-8728-4b3e-a301-fb8038dc5147/node-ca/0.log" Jul 02 14:33:58.886180 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:58.886145 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2xppm_81787771-8105-4b8d-bec6-3ad8cca4ef30/serve-healthcheck-canary/0.log" Jul 02 14:33:59.316914 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:59.316864 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lh2qk_d510c8d6-85b6-4dce-ae37-9cb571bf76f6/kube-rbac-proxy/0.log" Jul 02 14:33:59.342286 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:59.342246 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lh2qk_d510c8d6-85b6-4dce-ae37-9cb571bf76f6/exporter/0.log" Jul 02 14:33:59.369239 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:33:59.369198 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lh2qk_d510c8d6-85b6-4dce-ae37-9cb571bf76f6/extractor/0.log" Jul 02 14:34:01.755295 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:01.755256 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-6b2rb_294774ae-0e9f-4b06-ad78-07d5b880c792/manager/0.log" Jul 02 14:34:01.792698 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:01.792666 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-hgdjc_6b0600b7-b349-464c-9b85-b9963831ad20/s3-init/0.log" Jul 02 14:34:01.813596 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:01.813565 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-hgdjc_6b0600b7-b349-464c-9b85-b9963831ad20/download-hf-model/0.log" Jul 02 14:34:02.553899 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:02.553873 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-kqwrh" Jul 02 14:34:06.420854 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:06.420821 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-pfgpk_6dd73fff-e40d-4e10-a678-2b6439f414db/kube-storage-version-migrator-operator/1.log" Jul 02 14:34:06.421691 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:06.421675 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-pfgpk_6dd73fff-e40d-4e10-a678-2b6439f414db/kube-storage-version-migrator-operator/0.log" Jul 02 14:34:07.409482 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:07.409388 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9vc4r_d7ae3d5a-ec79-41a6-8ae7-fb54578f26d3/kube-multus/0.log" Jul 02 14:34:07.433838 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:07.433817 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f8xxj_5d4fe52c-af75-465b-906e-3703bc2042ce/kube-multus-additional-cni-plugins/0.log" Jul 02 14:34:07.456011 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:07.455985 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f8xxj_5d4fe52c-af75-465b-906e-3703bc2042ce/egress-router-binary-copy/0.log" Jul 02 14:34:07.478079 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:07.478030 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f8xxj_5d4fe52c-af75-465b-906e-3703bc2042ce/cni-plugins/0.log" Jul 02 14:34:07.500121 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:07.500094 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f8xxj_5d4fe52c-af75-465b-906e-3703bc2042ce/bond-cni-plugin/0.log" Jul 02 14:34:07.522713 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:07.522692 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f8xxj_5d4fe52c-af75-465b-906e-3703bc2042ce/routeoverride-cni/0.log" Jul 02 14:34:07.547907 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:07.547873 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f8xxj_5d4fe52c-af75-465b-906e-3703bc2042ce/whereabouts-cni-bincopy/0.log" Jul 02 14:34:07.570492 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:07.570467 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f8xxj_5d4fe52c-af75-465b-906e-3703bc2042ce/whereabouts-cni/0.log" Jul 02 14:34:08.031683 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:08.031652 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p8qsr_48976aeb-7f1a-4164-a77b-9bdc07d79cb9/network-metrics-daemon/0.log" Jul 02 14:34:08.052929 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:08.052908 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p8qsr_48976aeb-7f1a-4164-a77b-9bdc07d79cb9/kube-rbac-proxy/0.log" Jul 02 14:34:09.270434 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:09.270400 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-controller/0.log" Jul 02 14:34:09.292667 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:09.292641 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/1.log" Jul 02 14:34:09.292816 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:09.292644 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovn-acl-logging/0.log" Jul 02 14:34:09.314386 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:09.314362 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/kube-rbac-proxy-node/0.log" Jul 02 14:34:09.340056 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:09.340022 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/kube-rbac-proxy-ovn-metrics/0.log" Jul 02 14:34:09.360726 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:09.360704 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/northd/0.log" Jul 02 14:34:09.383827 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:09.383804 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/nbdb/0.log" Jul 02 14:34:09.406206 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:09.406184 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/sbdb/0.log" Jul 02 14:34:09.452533 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:09.452502 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9gx2_bdbc7835-070d-4af2-987c-987dd8ab81fd/ovnkube-controller/0.log" Jul 02 14:34:10.423095 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:10.423067 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-56499946bc-5khc9_6b5e649e-89d0-4f1c-a245-028b1900bc9b/check-endpoints/0.log" Jul 02 14:34:10.445941 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:10.445918 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-jn2xt_4ef79fd6-3648-4b4b-bd70-38b27b45aeb5/network-check-target-container/0.log" Jul 02 14:34:11.386152 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:11.386120 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-rfk79_c30d5842-ae96-42d5-8074-d4f321b23818/iptables-alerter/0.log" Jul 02 14:34:11.457114 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:11.457086 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmznz_2081cca0-7355-4c02-85dc-b959914fbf47/konnectivity-agent/0.log" limit=1024 Jul 02 14:34:12.065796 ip-10-0-137-40 kubenswrapper[2561]: I0702 14:34:12.065768 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-q98ws_caa799d4-3d86-4b19-9af0-2d4b19f82cb4/tuned/0.log"