Jul 07 22:02:09.295547 ip-10-0-129-68 systemd[1]: Starting Kubernetes Kubelet... Jul 07 22:02:09.679047 ip-10-0-129-68 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 22:02:09.679047 ip-10-0-129-68 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 22:02:09.679047 ip-10-0-129-68 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 22:02:09.679047 ip-10-0-129-68 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 22:02:09.679047 ip-10-0-129-68 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 22:02:09.679845 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.679117 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 22:02:09.684596 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684575 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:02:09.684596 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684592 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:02:09.684596 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684597 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:02:09.684596 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684601 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:02:09.684596 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684605 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684609 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684613 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684617 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684631 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684635 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684639 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684642 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684646 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684652 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684655 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684659 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684663 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684667 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684671 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684674 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684678 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684681 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684685 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684689 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:02:09.684885 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684693 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684696 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684700 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684703 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684707 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684710 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684714 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684718 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684722 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684726 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684732 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684739 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684743 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684747 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684755 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684761 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684766 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684771 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684776 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:02:09.685771 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684781 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684785 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684789 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684794 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684798 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684802 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684805 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684809 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684822 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684827 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684831 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684836 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684840 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684844 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684848 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684852 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684857 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684861 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684865 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684869 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:02:09.686602 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684873 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684877 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684881 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684886 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684889 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684893 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684897 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684902 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684907 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684912 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684917 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684922 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684926 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684930 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684934 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684938 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684942 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684946 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684951 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684955 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:02:09.687626 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684959 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684964 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684968 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684972 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684976 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684981 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684985 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684989 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684993 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.684997 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.685001 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.685007 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.685012 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.685017 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.685022 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.685026 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686170 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686179 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686184 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:02:09.688505 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686190 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686195 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686200 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686205 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686209 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686215 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686219 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686224 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686228 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686255 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686260 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686264 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686268 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686273 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686277 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686281 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686285 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686289 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686293 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686297 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:02:09.689265 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686300 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686304 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686308 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686312 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686316 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686320 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686324 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686328 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686332 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686336 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686341 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686345 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686350 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686354 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686358 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686362 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686366 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686371 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686376 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686381 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:02:09.689896 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686385 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686391 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686395 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686400 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686403 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686407 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686411 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686416 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686420 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686424 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686428 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686432 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686436 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686441 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686445 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686449 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686453 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686457 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686461 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686465 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:02:09.690408 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686469 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686473 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686479 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686486 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686494 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686501 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686507 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686511 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686520 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686526 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686531 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686536 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686540 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686544 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686549 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686553 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686557 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686561 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686565 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:02:09.690909 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686569 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686573 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686578 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686582 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686586 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686589 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686594 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686598 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686601 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686605 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686609 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686613 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686617 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686621 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686625 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686629 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.686633 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687182 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687190 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:02:09.691396 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687195 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687199 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687203 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687206 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687210 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687215 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687218 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687223 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687227 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687253 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687257 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687262 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687266 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687270 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687274 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687278 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687281 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687286 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687290 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687294 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:02:09.691856 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687299 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687304 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687308 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687312 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687316 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687320 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687324 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687328 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687332 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687336 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687340 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687346 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687351 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687356 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687360 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687364 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687368 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687372 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687377 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687381 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:02:09.692368 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687385 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687390 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687394 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687398 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687402 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687406 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687410 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687415 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687418 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687422 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687427 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687431 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687436 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687440 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687444 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687448 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687452 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687456 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687460 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687464 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:02:09.692847 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687469 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687473 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687477 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687481 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687488 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687493 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687498 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687503 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687507 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687511 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687516 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687521 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687525 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687529 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687533 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687537 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687541 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687545 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687549 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:02:09.693371 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687553 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687557 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687561 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687565 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687569 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687574 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687578 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687582 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687588 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687596 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687600 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687604 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687609 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687616 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687621 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687625 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687630 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.687634 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687722 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687732 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 22:02:09.693817 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687742 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687749 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687755 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687760 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687766 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687772 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687777 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687782 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687787 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687792 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687797 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687801 2580 flags.go:64] FLAG: --cgroup-root="" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687805 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687810 2580 flags.go:64] FLAG: --client-ca-file="" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687815 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687819 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687825 2580 flags.go:64] FLAG: --cluster-domain="" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687830 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687835 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687866 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687873 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687880 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687885 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 22:02:09.694342 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687889 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687894 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687899 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687903 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687907 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687912 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687917 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687923 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687928 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687932 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687937 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687942 2580 flags.go:64] FLAG: --enable-server="true" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687947 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687954 2580 flags.go:64] FLAG: --event-burst="100" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687959 2580 flags.go:64] FLAG: --event-qps="50" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687964 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687969 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687973 2580 flags.go:64] FLAG: --eviction-hard="" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687979 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687983 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687988 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687992 2580 flags.go:64] FLAG: --eviction-soft="" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.687997 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688002 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688006 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 22:02:09.694886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688011 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688015 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688020 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688024 2580 flags.go:64] FLAG: --feature-gates="" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688030 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688035 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688040 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688045 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688050 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688054 2580 flags.go:64] FLAG: --help="false" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688059 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-129-68.ec2.internal" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688064 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688068 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688073 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688079 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688085 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688089 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688094 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688098 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688103 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688107 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688112 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688117 2580 flags.go:64] FLAG: --kube-reserved="" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688122 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 22:02:09.695560 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688127 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688132 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688136 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688141 2580 flags.go:64] FLAG: --lock-file="" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688145 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688150 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688155 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688163 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688168 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688173 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688177 2580 flags.go:64] FLAG: --logging-format="text" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688182 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688187 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688192 2580 flags.go:64] FLAG: --manifest-url="" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688196 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688203 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688207 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688213 2580 flags.go:64] FLAG: --max-pods="110" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688218 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688223 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688227 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688250 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688256 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688260 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688265 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 22:02:09.696138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688277 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688282 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688286 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688291 2580 flags.go:64] FLAG: --pod-cidr="" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688295 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688303 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688308 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688313 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688318 2580 flags.go:64] FLAG: --port="10250" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688323 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688327 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0233ac9372985be4c" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688332 2580 flags.go:64] FLAG: --qos-reserved="" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688337 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688341 2580 flags.go:64] FLAG: --register-node="true" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688345 2580 flags.go:64] FLAG: --register-with-taints="" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688351 2580 flags.go:64] FLAG: --registry-burst="10" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688356 2580 flags.go:64] FLAG: --registry-qps="5" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688360 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688364 2580 flags.go:64] FLAG: --reserved-memory="" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688370 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688375 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688379 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688384 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688388 2580 flags.go:64] FLAG: --runonce="false" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688393 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 22:02:09.696776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688398 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688402 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688407 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688411 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688416 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688420 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688425 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688432 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688437 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688441 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688446 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688450 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688455 2580 flags.go:64] FLAG: --system-cgroups="" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688459 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688467 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688473 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688477 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688483 2580 flags.go:64] FLAG: --tls-min-version="" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688487 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688491 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688496 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688500 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688505 2580 flags.go:64] FLAG: --v="2" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688510 2580 flags.go:64] FLAG: --version="false" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688516 2580 flags.go:64] FLAG: --vmodule="" Jul 07 22:02:09.697415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688521 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.688526 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.689953 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690088 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690093 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690097 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690100 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690103 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690106 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690109 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690111 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690114 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690117 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690119 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690122 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690126 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690130 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690133 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690136 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690139 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:02:09.698017 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690142 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690145 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690147 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690150 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690153 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690155 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690158 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690160 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690163 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690165 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690168 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690170 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690173 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690175 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690180 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690187 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690190 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690192 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690195 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:02:09.698543 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690198 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690200 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690203 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690205 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690208 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690210 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690213 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690215 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690217 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690220 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690222 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690225 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690227 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690229 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690232 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690254 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690256 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690259 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690261 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690264 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:02:09.699032 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690267 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690269 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690271 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690274 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690277 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690279 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690282 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690286 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690291 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690293 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690296 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690299 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690301 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690303 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690306 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690309 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690313 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690317 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690320 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:02:09.699534 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690322 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690325 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690327 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690330 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690332 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690335 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690337 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690340 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690342 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690345 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690347 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690349 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690352 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690354 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690356 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690359 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690361 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690364 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690366 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:02:09.700005 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690370 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690375 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690379 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.690382 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.690388 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.697598 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.697612 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.697631 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.697640 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697691 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697696 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697699 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697702 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697705 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697708 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:02:09.700483 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697711 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697714 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697717 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697720 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697722 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697725 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697727 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697730 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697734 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697738 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697740 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697743 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697745 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697748 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697750 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697753 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697755 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697758 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697760 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:02:09.700864 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697763 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697765 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697768 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697771 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697775 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697777 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697780 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697782 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697785 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697787 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697790 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697792 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697794 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697797 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697799 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697802 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697804 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697806 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697809 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697811 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:02:09.701349 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697813 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697815 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697818 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697820 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697823 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697825 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697827 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697830 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697832 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697834 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697836 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697839 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697842 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697846 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697849 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697852 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697855 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697858 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697860 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697863 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:02:09.701832 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697865 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697867 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697870 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697874 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697877 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697879 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697882 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697884 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697887 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697891 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697894 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697896 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697899 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697901 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697904 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697906 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697908 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697911 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697913 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:02:09.702436 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697915 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697918 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697920 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697922 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697925 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697927 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697930 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697932 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697936 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697938 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697941 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697944 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697946 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697949 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.697951 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:02:09.702894 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.697956 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698065 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698070 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698073 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698076 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698079 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698081 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698085 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698087 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698089 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698092 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698094 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698097 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698099 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698102 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698104 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698106 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698109 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698111 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698113 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:02:09.703266 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698116 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698118 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698120 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698123 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698125 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698128 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698131 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698133 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698136 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698139 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698141 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698143 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698146 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698148 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698151 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698153 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698155 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698158 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698160 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698162 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:02:09.703736 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698164 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698167 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698169 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698172 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698174 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698178 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698182 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698185 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698188 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698190 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698193 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698196 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698198 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698201 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698204 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698206 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698209 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698211 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:02:09.704273 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698214 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698216 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698219 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698222 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698225 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698227 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698229 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698232 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698251 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698254 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698256 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698259 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698261 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698264 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698266 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698268 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698271 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698273 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698276 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698278 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:02:09.704717 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698281 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698283 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698286 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698288 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698290 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698293 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698295 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698298 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698300 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698302 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698305 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698307 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698310 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698312 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698315 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698317 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698319 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698323 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698326 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698328 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:02:09.705228 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698331 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:02:09.705763 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:09.698334 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:02:09.705763 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.698339 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 22:02:09.705763 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.698434 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 22:02:09.705763 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.703844 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 22:02:09.705763 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.705286 2580 server.go:1019] "Starting client certificate rotation" Jul 07 22:02:09.705763 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.705395 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 22:02:09.705763 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.705428 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 22:02:09.707635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.707623 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 22:02:09.710911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.710891 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 22:02:09.721078 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.721061 2580 log.go:25] "Validated CRI v1 runtime API" Jul 07 22:02:09.723270 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.723256 2580 log.go:25] "Validated CRI v1 image API" Jul 07 22:02:09.723998 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.723976 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 22:02:09.726340 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.726322 2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d7714a2c-9df8-477d-b133-6ea41b3087dd:/dev/nvme0n1p3 d9a7894a-318c-4727-b9c6-49e44abe8c0c:/dev/nvme0n1p4] Jul 07 22:02:09.726408 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.726339 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 22:02:09.729221 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.729195 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 22:02:09.732698 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.732562 2580 manager.go:216] Machine: {Timestamp:2026-07-07 22:02:09.731306043 +0000 UTC m=+0.320684714 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096242 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26b52d5574210429fcc690ec95e3e7 SystemUUID:ec26b52d-5574-2104-29fc-c690ec95e3e7 BootID:8300bb66-83f3-4da0-bb87-7c1032e2bcb3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2c:6c:e2:05:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2c:6c:e2:05:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:09:e4:17:cc:80 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 22:02:09.732698 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.732688 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 22:02:09.732829 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.732759 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 22:02:09.735966 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.735940 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 22:02:09.736092 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.735967 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-68.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 22:02:09.736692 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.736682 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 22:02:09.736725 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.736695 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 22:02:09.736725 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.736702 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 22:02:09.737400 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.737389 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 22:02:09.739496 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.739485 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 22:02:09.742842 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.742830 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 07 22:02:09.743101 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.743091 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 22:02:09.743135 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.743122 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 07 22:02:09.743135 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.743132 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 22:02:09.743263 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.743143 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 22:02:09.743263 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.743152 2580 kubelet.go:398] "Adding apiserver pod source" Jul 07 22:02:09.743263 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.743160 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 22:02:09.743474 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.743462 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 22:02:09.743526 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.743481 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 22:02:09.750649 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.750635 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 22:02:09.752830 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.752807 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 22:02:09.752884 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.752847 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 22:02:09.754296 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754285 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 22:02:09.754337 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754302 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 22:02:09.754337 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754308 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 22:02:09.754337 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754314 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 22:02:09.754337 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754323 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 22:02:09.754337 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754328 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 22:02:09.754337 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754336 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 22:02:09.754498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754341 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 22:02:09.754498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754347 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 22:02:09.754498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754353 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 22:02:09.754498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754368 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 22:02:09.754498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754373 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 22:02:09.754498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.754482 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l4dc5" Jul 07 22:02:09.755135 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.755125 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 22:02:09.755135 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.755135 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 22:02:09.755191 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:09.755156 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 22:02:09.755191 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:09.755156 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 22:02:09.758106 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.758093 2580 server.go:1268] "Started kubelet" Jul 07 22:02:09.758231 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.758158 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 22:02:09.758383 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.758213 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 22:02:09.758473 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.758410 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 22:02:09.758932 ip-10-0-129-68 systemd[1]: Started Kubernetes Kubelet. Jul 07 22:02:09.760353 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.760334 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 07 22:02:09.760445 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.760336 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 22:02:09.761080 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.761059 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l4dc5" Jul 07 22:02:09.763309 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:09.763289 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 22:02:09.768016 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.767997 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 22:02:09.768016 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.768016 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 22:02:09.768591 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:09.768566 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-68.ec2.internal\" not found" Jul 07 22:02:09.768705 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.768693 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 22:02:09.768793 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.768776 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 22:02:09.768793 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.768692 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 22:02:09.768936 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.768894 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 07 22:02:09.768936 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.768902 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 07 22:02:09.769330 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.769314 2580 factory.go:153] Registering CRI-O factory Jul 07 22:02:09.769330 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.769331 2580 factory.go:223] Registration of the crio container factory successfully Jul 07 22:02:09.769451 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.769372 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 22:02:09.769451 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.769381 2580 factory.go:55] Registering systemd factory Jul 07 22:02:09.769451 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.769389 2580 factory.go:223] Registration of the systemd container factory successfully Jul 07 22:02:09.769451 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.769408 2580 factory.go:103] Registering Raw factory Jul 07 22:02:09.769451 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.769418 2580 manager.go:1196] Started watching for new ooms in manager Jul 07 22:02:09.769948 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.769934 2580 manager.go:349] Starting recovery of all containers Jul 07 22:02:09.770119 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.770098 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:02:09.776378 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.776334 2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-68.ec2.internal" not found Jul 07 22:02:09.776479 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:09.776413 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-68.ec2.internal\" not found" node="ip-10-0-129-68.ec2.internal" Jul 07 22:02:09.783479 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.783375 2580 manager.go:354] Recovery completed Jul 07 22:02:09.787477 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.787464 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:02:09.789358 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.789343 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:02:09.789424 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.789371 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:02:09.789424 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.789382 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:02:09.789876 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.789863 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 22:02:09.789928 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.789877 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 22:02:09.789928 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.789898 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 07 22:02:09.789928 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.789904 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:02:09.790524 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.790512 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:02:09.790580 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.790538 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:02:09.790580 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.790548 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:02:09.791231 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.791219 2580 policy_none.go:49] "None policy: Start" Jul 07 22:02:09.791307 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.791281 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 22:02:09.791307 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.791298 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 22:02:09.792210 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.792200 2580 policy_none.go:47] "Start" Jul 07 22:02:09.792264 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.792221 2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-68.ec2.internal" not found Jul 07 22:02:09.823823 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.823802 2580 manager.go:337] "Starting Device Plugin manager" Jul 07 22:02:09.842093 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:09.823834 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 22:02:09.842093 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.823843 2580 server.go:87] "Starting device plugin registration server" Jul 07 22:02:09.842093 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.824053 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 22:02:09.842093 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.824068 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 22:02:09.842093 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.824158 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 22:02:09.842093 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.824222 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 22:02:09.842093 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.824230 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 22:02:09.842093 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:09.824742 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 22:02:09.842093 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:09.824784 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-68.ec2.internal\" not found" Jul 07 22:02:09.850340 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.850325 2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-68.ec2.internal" not found Jul 07 22:02:09.924598 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.924573 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:02:09.925789 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.925761 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:02:09.925903 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.925803 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:02:09.925903 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.925821 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:02:09.925903 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.925855 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-68.ec2.internal" Jul 07 22:02:09.926586 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.926556 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 22:02:09.928014 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.927996 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 22:02:09.928081 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.928042 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 22:02:09.928081 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.928062 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 22:02:09.928161 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:09.928091 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 22:02:09.930106 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.930051 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:02:09.934889 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:09.934873 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-68.ec2.internal" Jul 07 22:02:09.934971 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:09.934893 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-68.ec2.internal\": node \"ip-10-0-129-68.ec2.internal\" not found" Jul 07 22:02:09.975212 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:09.975189 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-68.ec2.internal\" not found" Jul 07 22:02:10.028294 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.028270 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-68.ec2.internal"] Jul 07 22:02:10.028366 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.028346 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:02:10.030007 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.029993 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:02:10.030058 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.030022 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:02:10.030058 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.030031 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:02:10.031536 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.031525 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:02:10.031669 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.031653 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.031705 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.031692 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:02:10.032223 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.032207 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:02:10.032322 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.032207 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:02:10.032322 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.032255 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:02:10.032322 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.032266 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:02:10.032322 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.032271 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:02:10.032322 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.032276 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:02:10.033772 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.033755 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.033854 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.033788 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:02:10.034442 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.034426 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:02:10.034515 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.034459 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:02:10.034515 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.034468 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:02:10.065851 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.065833 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-68.ec2.internal\" not found" node="ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.070164 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.070146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2cc3ad308bf9860b5b3bfada611aed6f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-68.ec2.internal\" (UID: \"2cc3ad308bf9860b5b3bfada611aed6f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.070281 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.070175 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4aebacaa3d93a6d5bf1c346f5aa8f73-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal\" (UID: \"d4aebacaa3d93a6d5bf1c346f5aa8f73\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.070281 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.070195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4aebacaa3d93a6d5bf1c346f5aa8f73-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal\" (UID: \"d4aebacaa3d93a6d5bf1c346f5aa8f73\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.070281 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.070200 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-68.ec2.internal\" not found" node="ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.075285 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.075270 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-68.ec2.internal\" not found" Jul 07 22:02:10.170500 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.170472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4aebacaa3d93a6d5bf1c346f5aa8f73-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal\" (UID: \"d4aebacaa3d93a6d5bf1c346f5aa8f73\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.170500 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.170502 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4aebacaa3d93a6d5bf1c346f5aa8f73-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal\" (UID: \"d4aebacaa3d93a6d5bf1c346f5aa8f73\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.170664 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.170525 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2cc3ad308bf9860b5b3bfada611aed6f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-68.ec2.internal\" (UID: \"2cc3ad308bf9860b5b3bfada611aed6f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.170664 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.170559 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4aebacaa3d93a6d5bf1c346f5aa8f73-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal\" (UID: \"d4aebacaa3d93a6d5bf1c346f5aa8f73\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.170664 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.170576 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2cc3ad308bf9860b5b3bfada611aed6f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-68.ec2.internal\" (UID: \"2cc3ad308bf9860b5b3bfada611aed6f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.170664 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.170579 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4aebacaa3d93a6d5bf1c346f5aa8f73-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal\" (UID: \"d4aebacaa3d93a6d5bf1c346f5aa8f73\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.175595 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.175579 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-68.ec2.internal\" not found" Jul 07 22:02:10.276368 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.276304 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-68.ec2.internal\" not found" Jul 07 22:02:10.370696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.370662 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.374334 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.374306 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.376392 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.376376 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-68.ec2.internal\" not found" Jul 07 22:02:10.476880 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.476852 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-68.ec2.internal\" not found" Jul 07 22:02:10.577340 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.577318 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-68.ec2.internal\" not found" Jul 07 22:02:10.677953 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.677929 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-68.ec2.internal\" not found" Jul 07 22:02:10.688640 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.688622 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:02:10.704450 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.704433 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 22:02:10.704538 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.704522 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 22:02:10.704593 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.704573 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 22:02:10.704668 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.704573 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 22:02:10.737296 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.736177 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:02:10.744259 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.744228 2580 apiserver.go:52] "Watching apiserver" Jul 07 22:02:10.750889 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.750873 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 22:02:10.751285 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.751255 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-2hj4g","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65","openshift-dns/node-resolver-hfthf","openshift-image-registry/node-ca-7sxcg","openshift-multus/multus-wq9vl","openshift-multus/network-metrics-daemon-k9ljx","openshift-network-diagnostics/network-check-target-qsxfk","openshift-network-operator/iptables-alerter-glszg","kube-system/global-pull-secret-syncer-59xcs","openshift-cluster-node-tuning-operator/tuned-b4fxx","openshift-multus/multus-additional-cni-plugins-zd4nf","openshift-ovn-kubernetes/ovnkube-node-hk92b"] Jul 07 22:02:10.753177 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.753160 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hfthf" Jul 07 22:02:10.754502 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.754482 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7sxcg" Jul 07 22:02:10.755108 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.755090 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 22:02:10.755198 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.755105 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 22:02:10.755198 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.755131 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ccxxt\"" Jul 07 22:02:10.755567 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.755555 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:10.755628 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.755606 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:10.756108 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.756095 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 22:02:10.756306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.756271 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 22:02:10.756403 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.756389 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 22:02:10.756661 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.756639 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5hrqv\"" Jul 07 22:02:10.756786 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.756773 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.757956 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.757941 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:10.758371 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.758347 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 22:02:10.758443 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.758371 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 22:02:10.758525 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.758508 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s9xll\"" Jul 07 22:02:10.759188 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.759174 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.759600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.759580 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 22:02:10.759704 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.759690 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5c29n\"" Jul 07 22:02:10.759772 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.759691 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 22:02:10.760582 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.760569 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.760687 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.760675 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 22:02:10.761011 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.760997 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 22:02:10.761136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.761126 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 22:02:10.761213 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.761126 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-254xl\"" Jul 07 22:02:10.762111 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.762095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:10.762197 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.762148 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:10.762467 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.762454 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 22:02:10.762835 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.762811 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 22:02:10.762917 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.762875 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 22:02:10.762983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.762937 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 22:02:10.763139 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.763124 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7bfks\"" Jul 07 22:02:10.763476 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.763459 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-glszg" Jul 07 22:02:10.764966 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.764949 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:10.765037 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.765011 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:10.765170 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.765153 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 22:02:10.765170 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.765163 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 22:02:10.765361 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.765296 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 22:02:10.765361 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.765362 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gcwtq\"" Jul 07 22:02:10.766341 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.766327 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.766575 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.766508 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 21:57:10 +0000 UTC" deadline="2028-04-16 01:51:39.260550868 +0000 UTC" Jul 07 22:02:10.766656 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.766579 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15555h49m28.493976162s" Jul 07 22:02:10.767658 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.767641 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.768379 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.768365 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.768456 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.768348 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 22:02:10.768575 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.768558 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 22:02:10.768631 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.768561 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 22:02:10.771513 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.771489 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dkttv\"" Jul 07 22:02:10.771959 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.771944 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 22:02:10.772165 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.772148 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 22:02:10.772477 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.772459 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 22:02:10.772883 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.772747 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4sdsc\"" Jul 07 22:02:10.774007 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.773082 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 22:02:10.774007 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.773181 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 22:02:10.774007 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.773348 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 22:02:10.774007 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.773465 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 22:02:10.774388 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774371 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-host\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.774441 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774403 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ae3c8757-074f-43f8-9f32-5a69e78843b9-cni-binary-copy\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.774441 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774427 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-socket-dir-parent\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.774556 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774448 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-etc-openvswitch\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.774556 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774468 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c41ea711-d80d-4533-a02f-5328426cedb9-serviceca\") pod \"node-ca-7sxcg\" (UID: \"c41ea711-d80d-4533-a02f-5328426cedb9\") " pod="openshift-image-registry/node-ca-7sxcg" Jul 07 22:02:10.774556 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774488 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0fc51f9e-2d89-4649-8992-87f23e1e7721-iptables-alerter-script\") pod \"iptables-alerter-glszg\" (UID: \"0fc51f9e-2d89-4649-8992-87f23e1e7721\") " pod="openshift-network-operator/iptables-alerter-glszg" Jul 07 22:02:10.774556 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774508 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sthr5\" (UniqueName: \"kubernetes.io/projected/eaf8d762-016e-43f4-bcce-ffd737a1641a-kube-api-access-sthr5\") pod \"node-resolver-hfthf\" (UID: \"eaf8d762-016e-43f4-bcce-ffd737a1641a\") " pod="openshift-dns/node-resolver-hfthf" Jul 07 22:02:10.774556 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbxmt\" (UniqueName: \"kubernetes.io/projected/af520e56-7ba8-4259-a907-7a5a28c41e5b-kube-api-access-rbxmt\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:10.774828 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774589 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-sysctl-d\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.774828 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774619 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2fk8\" (UniqueName: \"kubernetes.io/projected/43af0cd5-5dfe-45f2-a50e-84bcda42d705-kube-api-access-x2fk8\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.774828 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774643 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-cni-dir\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.774828 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774665 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-var-lib-cni-multus\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.774828 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774688 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-var-lib-kubelet\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.774828 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-socket-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.774828 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774742 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0fc51f9e-2d89-4649-8992-87f23e1e7721-host-slash\") pod \"iptables-alerter-glszg\" (UID: \"0fc51f9e-2d89-4649-8992-87f23e1e7721\") " pod="openshift-network-operator/iptables-alerter-glszg" Jul 07 22:02:10.774828 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774772 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-daemon-config\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.774828 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774799 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-kubelet\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.774828 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774822 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-systemd-units\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774855 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-etc-kubernetes\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774879 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7d5w\" (UniqueName: \"kubernetes.io/projected/ae3c8757-074f-43f8-9f32-5a69e78843b9-kube-api-access-j7d5w\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774902 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774917 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-sys-fs\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774931 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-modprobe-d\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774945 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-sysctl-conf\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774980 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-lib-modules\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.774998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-cni-bin\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775012 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-run-ovn\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c41ea711-d80d-4533-a02f-5328426cedb9-host\") pod \"node-ca-7sxcg\" (UID: \"c41ea711-d80d-4533-a02f-5328426cedb9\") " pod="openshift-image-registry/node-ca-7sxcg" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-var-lib-kubelet\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c272d87-a60e-4d9f-b942-655bbcb9b788-agent-certs\") pod \"konnectivity-agent-2hj4g\" (UID: \"5c272d87-a60e-4d9f-b942-655bbcb9b788\") " pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775084 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-run-multus-certs\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-os-release\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775126 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-ovnkube-config\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-cni-netd\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.775306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775160 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-tuned\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775191 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-device-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775209 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-sysconfig\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775222 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-cnibin\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-system-cni-dir\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775284 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775313 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaf8d762-016e-43f4-bcce-ffd737a1641a-tmp-dir\") pod \"node-resolver-hfthf\" (UID: \"eaf8d762-016e-43f4-bcce-ffd737a1641a\") " pod="openshift-dns/node-resolver-hfthf" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775353 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-systemd\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-var-lib-cni-bin\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775416 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-run-netns\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775440 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxk2l\" (UniqueName: \"kubernetes.io/projected/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-kube-api-access-qxk2l\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775461 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-run\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43af0cd5-5dfe-45f2-a50e-84bcda42d705-tmp\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775541 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-run-k8s-cni-cncf-io\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-conf-dir\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.776032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eaf8d762-016e-43f4-bcce-ffd737a1641a-hosts-file\") pod \"node-resolver-hfthf\" (UID: \"eaf8d762-016e-43f4-bcce-ffd737a1641a\") " pod="openshift-dns/node-resolver-hfthf" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775635 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60d1d68e-fcca-4c49-9d19-91608847c10b-kubelet-config\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775662 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60d1d68e-fcca-4c49-9d19-91608847c10b-dbus\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775690 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-cnibin\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775715 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7285ba25-8291-4c0a-acd4-b986c4652f41-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775757 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-slash\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775787 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-run-systemd\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775807 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-node-log\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775827 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-env-overrides\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775843 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-ovn-node-metrics-cert\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775856 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-ovnkube-script-lib\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlmzz\" (UniqueName: \"kubernetes.io/projected/0fc51f9e-2d89-4649-8992-87f23e1e7721-kube-api-access-rlmzz\") pod \"iptables-alerter-glszg\" (UID: \"0fc51f9e-2d89-4649-8992-87f23e1e7721\") " pod="openshift-network-operator/iptables-alerter-glszg" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775896 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-kubernetes\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-hostroot\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775944 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-log-socket\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-run-ovn-kubernetes\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776553 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.775993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c272d87-a60e-4d9f-b942-655bbcb9b788-konnectivity-ca\") pod \"konnectivity-agent-2hj4g\" (UID: \"5c272d87-a60e-4d9f-b942-655bbcb9b788\") " pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776016 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-registration-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpddz\" (UniqueName: \"kubernetes.io/projected/29d5e9a5-3040-4190-af5b-d9e97c3077ca-kube-api-access-qpddz\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776059 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-os-release\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776081 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-run-netns\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776094 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7285ba25-8291-4c0a-acd4-b986c4652f41-cni-binary-copy\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776107 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctp7b\" (UniqueName: \"kubernetes.io/projected/7285ba25-8291-4c0a-acd4-b986c4652f41-kube-api-access-ctp7b\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776122 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-var-lib-openvswitch\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776136 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-system-cni-dir\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776150 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-run-openvswitch\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzfw8\" (UniqueName: \"kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8\") pod \"network-check-target-qsxfk\" (UID: \"9b236333-c490-4f99-b91f-d44b01461c72\") " pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776182 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776196 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7285ba25-8291-4c0a-acd4-b986c4652f41-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776210 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wflwl\" (UniqueName: \"kubernetes.io/projected/c41ea711-d80d-4533-a02f-5328426cedb9-kube-api-access-wflwl\") pod \"node-ca-7sxcg\" (UID: \"c41ea711-d80d-4533-a02f-5328426cedb9\") " pod="openshift-image-registry/node-ca-7sxcg" Jul 07 22:02:10.776986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.776254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-sys\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.782850 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.782832 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 22:02:10.782956 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.782889 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-68.ec2.internal" Jul 07 22:02:10.783035 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.783014 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal"] Jul 07 22:02:10.784789 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.784769 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 22:02:10.789053 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.789039 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-68.ec2.internal"] Jul 07 22:02:10.789129 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.789117 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 22:02:10.808503 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.808485 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zp222" Jul 07 22:02:10.813856 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.813835 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zp222" Jul 07 22:02:10.876707 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876686 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2fk8\" (UniqueName: \"kubernetes.io/projected/43af0cd5-5dfe-45f2-a50e-84bcda42d705-kube-api-access-x2fk8\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.876793 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876714 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-cni-dir\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.876793 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-var-lib-cni-multus\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.876793 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876752 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-var-lib-kubelet\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.876884 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876798 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-var-lib-kubelet\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.876884 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876809 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-var-lib-cni-multus\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.876884 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876809 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-cni-dir\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.876884 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876829 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-socket-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.876884 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876858 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0fc51f9e-2d89-4649-8992-87f23e1e7721-host-slash\") pod \"iptables-alerter-glszg\" (UID: \"0fc51f9e-2d89-4649-8992-87f23e1e7721\") " pod="openshift-network-operator/iptables-alerter-glszg" Jul 07 22:02:10.876884 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876878 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-daemon-config\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-kubelet\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876923 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-socket-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876931 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0fc51f9e-2d89-4649-8992-87f23e1e7721-host-slash\") pod \"iptables-alerter-glszg\" (UID: \"0fc51f9e-2d89-4649-8992-87f23e1e7721\") " pod="openshift-network-operator/iptables-alerter-glszg" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-systemd-units\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876962 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-systemd-units\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876967 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-kubelet\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876972 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-etc-kubernetes\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.876991 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7d5w\" (UniqueName: \"kubernetes.io/projected/ae3c8757-074f-43f8-9f32-5a69e78843b9-kube-api-access-j7d5w\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877022 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-sys-fs\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877036 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-modprobe-d\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877038 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-etc-kubernetes\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-sysctl-conf\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-lib-modules\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.877136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877126 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-sys-fs\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877138 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877140 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-modprobe-d\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877168 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-cni-bin\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877185 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-run-ovn\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c41ea711-d80d-4533-a02f-5328426cedb9-host\") pod \"node-ca-7sxcg\" (UID: \"c41ea711-d80d-4533-a02f-5328426cedb9\") " pod="openshift-image-registry/node-ca-7sxcg" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877206 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-cni-bin\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877221 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-var-lib-kubelet\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c272d87-a60e-4d9f-b942-655bbcb9b788-agent-certs\") pod \"konnectivity-agent-2hj4g\" (UID: \"5c272d87-a60e-4d9f-b942-655bbcb9b788\") " pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877270 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-run-ovn\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877274 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-lib-modules\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877280 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-sysctl-conf\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-run-multus-certs\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877296 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-var-lib-kubelet\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877276 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c41ea711-d80d-4533-a02f-5328426cedb9-host\") pod \"node-ca-7sxcg\" (UID: \"c41ea711-d80d-4533-a02f-5328426cedb9\") " pod="openshift-image-registry/node-ca-7sxcg" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877323 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-os-release\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877385 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-run-multus-certs\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-ovnkube-config\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.877811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877424 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-cni-netd\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877431 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-os-release\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877422 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-daemon-config\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877469 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-cni-netd\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877498 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-tuned\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877527 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-device-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877585 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-sysconfig\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877600 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877612 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-cnibin\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877610 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-system-cni-dir\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877654 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-device-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877677 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-sysconfig\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877689 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaf8d762-016e-43f4-bcce-ffd737a1641a-tmp-dir\") pod \"node-resolver-hfthf\" (UID: \"eaf8d762-016e-43f4-bcce-ffd737a1641a\") " pod="openshift-dns/node-resolver-hfthf" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877680 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-cnibin\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877713 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:10.878635 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877692 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-system-cni-dir\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877802 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-systemd\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877851 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-var-lib-cni-bin\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.877876 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877918 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-systemd\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877928 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-var-lib-cni-bin\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877946 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-run-netns\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877878 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-run-netns\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877957 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaf8d762-016e-43f4-bcce-ffd737a1641a-tmp-dir\") pod \"node-resolver-hfthf\" (UID: \"eaf8d762-016e-43f4-bcce-ffd737a1641a\") " pod="openshift-dns/node-resolver-hfthf" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.877964 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-ovnkube-config\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.877990 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs podName:af520e56-7ba8-4259-a907-7a5a28c41e5b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:11.377944594 +0000 UTC m=+1.967323280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs") pod "network-metrics-daemon-k9ljx" (UID: "af520e56-7ba8-4259-a907-7a5a28c41e5b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878018 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxk2l\" (UniqueName: \"kubernetes.io/projected/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-kube-api-access-qxk2l\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878038 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-run\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43af0cd5-5dfe-45f2-a50e-84bcda42d705-tmp\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-run-k8s-cni-cncf-io\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878095 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-conf-dir\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.879554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eaf8d762-016e-43f4-bcce-ffd737a1641a-hosts-file\") pod \"node-resolver-hfthf\" (UID: \"eaf8d762-016e-43f4-bcce-ffd737a1641a\") " pod="openshift-dns/node-resolver-hfthf" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878132 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60d1d68e-fcca-4c49-9d19-91608847c10b-kubelet-config\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878155 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60d1d68e-fcca-4c49-9d19-91608847c10b-dbus\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878164 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-run\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878179 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-cnibin\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878198 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-conf-dir\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878206 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-run-k8s-cni-cncf-io\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878207 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7285ba25-8291-4c0a-acd4-b986c4652f41-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878261 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-slash\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878271 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60d1d68e-fcca-4c49-9d19-91608847c10b-kubelet-config\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878272 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eaf8d762-016e-43f4-bcce-ffd737a1641a-hosts-file\") pod \"node-resolver-hfthf\" (UID: \"eaf8d762-016e-43f4-bcce-ffd737a1641a\") " pod="openshift-dns/node-resolver-hfthf" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-run-systemd\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-node-log\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878314 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7285ba25-8291-4c0a-acd4-b986c4652f41-cnibin\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878330 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60d1d68e-fcca-4c49-9d19-91608847c10b-dbus\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878334 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-env-overrides\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878352 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-run-systemd\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878380 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-ovn-node-metrics-cert\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.880406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878389 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-slash\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-ovnkube-script-lib\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878431 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-node-log\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878470 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlmzz\" (UniqueName: \"kubernetes.io/projected/0fc51f9e-2d89-4649-8992-87f23e1e7721-kube-api-access-rlmzz\") pod \"iptables-alerter-glszg\" (UID: \"0fc51f9e-2d89-4649-8992-87f23e1e7721\") " pod="openshift-network-operator/iptables-alerter-glszg" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878496 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-kubernetes\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-hostroot\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-log-socket\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878690 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-run-ovn-kubernetes\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c272d87-a60e-4d9f-b942-655bbcb9b788-konnectivity-ca\") pod \"konnectivity-agent-2hj4g\" (UID: \"5c272d87-a60e-4d9f-b942-655bbcb9b788\") " pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878797 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-registration-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878822 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-env-overrides\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.878850 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpddz\" (UniqueName: \"kubernetes.io/projected/29d5e9a5-3040-4190-af5b-d9e97c3077ca-kube-api-access-qpddz\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-os-release\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-run-netns\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879032 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7285ba25-8291-4c0a-acd4-b986c4652f41-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879064 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7285ba25-8291-4c0a-acd4-b986c4652f41-cni-binary-copy\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctp7b\" (UniqueName: \"kubernetes.io/projected/7285ba25-8291-4c0a-acd4-b986c4652f41-kube-api-access-ctp7b\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.881320 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-var-lib-openvswitch\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879120 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-run-ovn-kubernetes\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879170 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-system-cni-dir\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879262 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-system-cni-dir\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879733 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-ovnkube-script-lib\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879801 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-run-openvswitch\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzfw8\" (UniqueName: \"kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8\") pod \"network-check-target-qsxfk\" (UID: \"9b236333-c490-4f99-b91f-d44b01461c72\") " pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879833 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c272d87-a60e-4d9f-b942-655bbcb9b788-konnectivity-ca\") pod \"konnectivity-agent-2hj4g\" (UID: \"5c272d87-a60e-4d9f-b942-655bbcb9b788\") " pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879857 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879874 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-kubernetes\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879932 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7285ba25-8291-4c0a-acd4-b986c4652f41-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879958 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880002 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wflwl\" (UniqueName: \"kubernetes.io/projected/c41ea711-d80d-4533-a02f-5328426cedb9-kube-api-access-wflwl\") pod \"node-ca-7sxcg\" (UID: \"c41ea711-d80d-4533-a02f-5328426cedb9\") " pod="openshift-image-registry/node-ca-7sxcg" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880042 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-sys\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-host\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ae3c8757-074f-43f8-9f32-5a69e78843b9-cni-binary-copy\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880115 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-os-release\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.882077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-socket-dir-parent\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880150 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-etc-openvswitch\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880165 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-host-run-netns\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880151 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-hostroot\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c41ea711-d80d-4533-a02f-5328426cedb9-serviceca\") pod \"node-ca-7sxcg\" (UID: \"c41ea711-d80d-4533-a02f-5328426cedb9\") " pod="openshift-image-registry/node-ca-7sxcg" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0fc51f9e-2d89-4649-8992-87f23e1e7721-iptables-alerter-script\") pod \"iptables-alerter-glszg\" (UID: \"0fc51f9e-2d89-4649-8992-87f23e1e7721\") " pod="openshift-network-operator/iptables-alerter-glszg" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sthr5\" (UniqueName: \"kubernetes.io/projected/eaf8d762-016e-43f4-bcce-ffd737a1641a-kube-api-access-sthr5\") pod \"node-resolver-hfthf\" (UID: \"eaf8d762-016e-43f4-bcce-ffd737a1641a\") " pod="openshift-dns/node-resolver-hfthf" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbxmt\" (UniqueName: \"kubernetes.io/projected/af520e56-7ba8-4259-a907-7a5a28c41e5b-kube-api-access-rbxmt\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-sysctl-d\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880468 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-sysctl-d\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880491 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-var-lib-openvswitch\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880525 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-log-socket\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.879933 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29d5e9a5-3040-4190-af5b-d9e97c3077ca-registration-dir\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880562 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-run-openvswitch\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880578 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.880769 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-sys\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.881801 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43af0cd5-5dfe-45f2-a50e-84bcda42d705-etc-tuned\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.882016 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43af0cd5-5dfe-45f2-a50e-84bcda42d705-tmp\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.882855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.882089 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-ovn-node-metrics-cert\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.883650 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.882259 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c272d87-a60e-4d9f-b942-655bbcb9b788-agent-certs\") pod \"konnectivity-agent-2hj4g\" (UID: \"5c272d87-a60e-4d9f-b942-655bbcb9b788\") " pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:10.883650 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.882283 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c41ea711-d80d-4533-a02f-5328426cedb9-serviceca\") pod \"node-ca-7sxcg\" (UID: \"c41ea711-d80d-4533-a02f-5328426cedb9\") " pod="openshift-image-registry/node-ca-7sxcg" Jul 07 22:02:10.883650 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.882393 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43af0cd5-5dfe-45f2-a50e-84bcda42d705-host\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.883650 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.882456 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0fc51f9e-2d89-4649-8992-87f23e1e7721-iptables-alerter-script\") pod \"iptables-alerter-glszg\" (UID: \"0fc51f9e-2d89-4649-8992-87f23e1e7721\") " pod="openshift-network-operator/iptables-alerter-glszg" Jul 07 22:02:10.883650 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.882467 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ae3c8757-074f-43f8-9f32-5a69e78843b9-multus-socket-dir-parent\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.883650 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.882529 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-etc-openvswitch\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.883650 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.882677 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7285ba25-8291-4c0a-acd4-b986c4652f41-cni-binary-copy\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.883650 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.882807 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:10.883650 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.882903 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7285ba25-8291-4c0a-acd4-b986c4652f41-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.883650 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.883009 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret podName:60d1d68e-fcca-4c49-9d19-91608847c10b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:11.38299 +0000 UTC m=+1.972368671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret") pod "global-pull-secret-syncer-59xcs" (UID: "60d1d68e-fcca-4c49-9d19-91608847c10b") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:10.883650 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.883103 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ae3c8757-074f-43f8-9f32-5a69e78843b9-cni-binary-copy\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.888223 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.888196 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctp7b\" (UniqueName: \"kubernetes.io/projected/7285ba25-8291-4c0a-acd4-b986c4652f41-kube-api-access-ctp7b\") pod \"multus-additional-cni-plugins-zd4nf\" (UID: \"7285ba25-8291-4c0a-acd4-b986c4652f41\") " pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:10.888349 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.888317 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:02:10.888349 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.888335 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:02:10.888349 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.888348 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lzfw8 for pod openshift-network-diagnostics/network-check-target-qsxfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:10.888511 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:10.888435 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8 podName:9b236333-c490-4f99-b91f-d44b01461c72 nodeName:}" failed. No retries permitted until 2026-07-07 22:02:11.388417676 +0000 UTC m=+1.977796356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lzfw8" (UniqueName: "kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8") pod "network-check-target-qsxfk" (UID: "9b236333-c490-4f99-b91f-d44b01461c72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:10.888886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.888833 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2fk8\" (UniqueName: \"kubernetes.io/projected/43af0cd5-5dfe-45f2-a50e-84bcda42d705-kube-api-access-x2fk8\") pod \"tuned-b4fxx\" (UID: \"43af0cd5-5dfe-45f2-a50e-84bcda42d705\") " pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:10.888886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.888919 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxk2l\" (UniqueName: \"kubernetes.io/projected/bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc-kube-api-access-qxk2l\") pod \"ovnkube-node-hk92b\" (UID: \"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:10.889681 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.889649 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpddz\" (UniqueName: \"kubernetes.io/projected/29d5e9a5-3040-4190-af5b-d9e97c3077ca-kube-api-access-qpddz\") pod \"aws-ebs-csi-driver-node-2jt65\" (UID: \"29d5e9a5-3040-4190-af5b-d9e97c3077ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:10.890328 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.890309 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7d5w\" (UniqueName: \"kubernetes.io/projected/ae3c8757-074f-43f8-9f32-5a69e78843b9-kube-api-access-j7d5w\") pod \"multus-wq9vl\" (UID: \"ae3c8757-074f-43f8-9f32-5a69e78843b9\") " pod="openshift-multus/multus-wq9vl" Jul 07 22:02:10.890685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.890662 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wflwl\" (UniqueName: \"kubernetes.io/projected/c41ea711-d80d-4533-a02f-5328426cedb9-kube-api-access-wflwl\") pod \"node-ca-7sxcg\" (UID: \"c41ea711-d80d-4533-a02f-5328426cedb9\") " pod="openshift-image-registry/node-ca-7sxcg" Jul 07 22:02:10.890762 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.890693 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbxmt\" (UniqueName: \"kubernetes.io/projected/af520e56-7ba8-4259-a907-7a5a28c41e5b-kube-api-access-rbxmt\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:10.891353 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.891336 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sthr5\" (UniqueName: \"kubernetes.io/projected/eaf8d762-016e-43f4-bcce-ffd737a1641a-kube-api-access-sthr5\") pod \"node-resolver-hfthf\" (UID: \"eaf8d762-016e-43f4-bcce-ffd737a1641a\") " pod="openshift-dns/node-resolver-hfthf" Jul 07 22:02:10.891627 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:10.891608 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlmzz\" (UniqueName: \"kubernetes.io/projected/0fc51f9e-2d89-4649-8992-87f23e1e7721-kube-api-access-rlmzz\") pod \"iptables-alerter-glszg\" (UID: \"0fc51f9e-2d89-4649-8992-87f23e1e7721\") " pod="openshift-network-operator/iptables-alerter-glszg" Jul 07 22:02:11.030402 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:11.030369 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc3ad308bf9860b5b3bfada611aed6f.slice/crio-9477d825c59ad9be7cf7e7195ebe02e8c17ec66cf1f30a86fb856185e8595fe6 WatchSource:0}: Error finding container 9477d825c59ad9be7cf7e7195ebe02e8c17ec66cf1f30a86fb856185e8595fe6: Status 404 returned error can't find the container with id 9477d825c59ad9be7cf7e7195ebe02e8c17ec66cf1f30a86fb856185e8595fe6 Jul 07 22:02:11.031448 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:11.031420 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4aebacaa3d93a6d5bf1c346f5aa8f73.slice/crio-4a1010a2aa45ec2105d78b453fe42ba070f5d0b30fa264149d82ffe7bef721b7 WatchSource:0}: Error finding container 4a1010a2aa45ec2105d78b453fe42ba070f5d0b30fa264149d82ffe7bef721b7: Status 404 returned error can't find the container with id 4a1010a2aa45ec2105d78b453fe42ba070f5d0b30fa264149d82ffe7bef721b7 Jul 07 22:02:11.035831 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.035816 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 22:02:11.080318 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.080294 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hfthf" Jul 07 22:02:11.087570 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:11.087540 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaf8d762_016e_43f4_bcce_ffd737a1641a.slice/crio-507cc4384b3284c27902677c252abb22a2cd037fd768477aebcd596f03345de2 WatchSource:0}: Error finding container 507cc4384b3284c27902677c252abb22a2cd037fd768477aebcd596f03345de2: Status 404 returned error can't find the container with id 507cc4384b3284c27902677c252abb22a2cd037fd768477aebcd596f03345de2 Jul 07 22:02:11.096434 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.096415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7sxcg" Jul 07 22:02:11.103007 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:11.102990 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41ea711_d80d_4533_a02f_5328426cedb9.slice/crio-52b1189edd1720ec49519141fb50bd51d789bd2839ec245dc0fdcfd11b01dbbe WatchSource:0}: Error finding container 52b1189edd1720ec49519141fb50bd51d789bd2839ec245dc0fdcfd11b01dbbe: Status 404 returned error can't find the container with id 52b1189edd1720ec49519141fb50bd51d789bd2839ec245dc0fdcfd11b01dbbe Jul 07 22:02:11.109809 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.109795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" Jul 07 22:02:11.118183 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:11.118164 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43af0cd5_5dfe_45f2_a50e_84bcda42d705.slice/crio-a3b2efe3a4b8a69bd8474c843289996e515667e7ecfc81c4350e66d35f8b9b34 WatchSource:0}: Error finding container a3b2efe3a4b8a69bd8474c843289996e515667e7ecfc81c4350e66d35f8b9b34: Status 404 returned error can't find the container with id a3b2efe3a4b8a69bd8474c843289996e515667e7ecfc81c4350e66d35f8b9b34 Jul 07 22:02:11.127974 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.127956 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:11.133742 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.133724 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" Jul 07 22:02:11.134525 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:11.134507 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c272d87_a60e_4d9f_b942_655bbcb9b788.slice/crio-1b3e2c2d5aa2cff1b2869f0861595f6925af3fefe7ff3b52876c2af8569fa209 WatchSource:0}: Error finding container 1b3e2c2d5aa2cff1b2869f0861595f6925af3fefe7ff3b52876c2af8569fa209: Status 404 returned error can't find the container with id 1b3e2c2d5aa2cff1b2869f0861595f6925af3fefe7ff3b52876c2af8569fa209 Jul 07 22:02:11.140343 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:11.140324 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29d5e9a5_3040_4190_af5b_d9e97c3077ca.slice/crio-743dc63b60117b466e09c9221e551e66c17f0b6dcde6355086221f01aa52eb45 WatchSource:0}: Error finding container 743dc63b60117b466e09c9221e551e66c17f0b6dcde6355086221f01aa52eb45: Status 404 returned error can't find the container with id 743dc63b60117b466e09c9221e551e66c17f0b6dcde6355086221f01aa52eb45 Jul 07 22:02:11.149284 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.149269 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wq9vl" Jul 07 22:02:11.155484 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:11.155462 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3c8757_074f_43f8_9f32_5a69e78843b9.slice/crio-222bf4d301fb7f711daa376bebecd7d80d68ef5df21cc2f14250d6b7c2b8405d WatchSource:0}: Error finding container 222bf4d301fb7f711daa376bebecd7d80d68ef5df21cc2f14250d6b7c2b8405d: Status 404 returned error can't find the container with id 222bf4d301fb7f711daa376bebecd7d80d68ef5df21cc2f14250d6b7c2b8405d Jul 07 22:02:11.164712 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.164694 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-glszg" Jul 07 22:02:11.170554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.170531 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zd4nf" Jul 07 22:02:11.171407 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:11.171386 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fc51f9e_2d89_4649_8992_87f23e1e7721.slice/crio-ab156bd5017fd238121f25ef9aeb3f4c587b63068524b976983369e93df116cd WatchSource:0}: Error finding container ab156bd5017fd238121f25ef9aeb3f4c587b63068524b976983369e93df116cd: Status 404 returned error can't find the container with id ab156bd5017fd238121f25ef9aeb3f4c587b63068524b976983369e93df116cd Jul 07 22:02:11.174354 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.174335 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:11.177897 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:11.177880 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7285ba25_8291_4c0a_acd4_b986c4652f41.slice/crio-4007992b87a16cfe33729ef9b362c44bb2fa43f8ed8c2b90476864162fa58166 WatchSource:0}: Error finding container 4007992b87a16cfe33729ef9b362c44bb2fa43f8ed8c2b90476864162fa58166: Status 404 returned error can't find the container with id 4007992b87a16cfe33729ef9b362c44bb2fa43f8ed8c2b90476864162fa58166 Jul 07 22:02:11.182321 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:02:11.182303 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc4474f8_d5cd_49ee_9618_cbf22bdbcbbc.slice/crio-1015a2f84e5823640cfcba721470b1455a6ab4e11599e8b6a284ddb5e0a04d12 WatchSource:0}: Error finding container 1015a2f84e5823640cfcba721470b1455a6ab4e11599e8b6a284ddb5e0a04d12: Status 404 returned error can't find the container with id 1015a2f84e5823640cfcba721470b1455a6ab4e11599e8b6a284ddb5e0a04d12 Jul 07 22:02:11.183857 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:11.183836 2580 kuberuntime_manager.go:1459] "Unhandled Error" err="container ovn-controller start failed in pod ovnkube-node-hk92b_openshift-ovn-kubernetes(bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 07 22:02:11.185281 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:11.185265 2580 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-rbac-proxy-node start failed in pod ovnkube-node-hk92b_openshift-ovn-kubernetes(bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 07 22:02:11.188178 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:11.188131 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" podUID="bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc" Jul 07 22:02:11.383690 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.383602 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:11.383690 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.383649 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:11.383847 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:11.383755 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:11.383847 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:11.383755 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:11.383847 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:11.383823 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret podName:60d1d68e-fcca-4c49-9d19-91608847c10b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:12.383802635 +0000 UTC m=+2.973181308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret") pod "global-pull-secret-syncer-59xcs" (UID: "60d1d68e-fcca-4c49-9d19-91608847c10b") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:11.383847 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:11.383839 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs podName:af520e56-7ba8-4259-a907-7a5a28c41e5b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:12.383831066 +0000 UTC m=+2.973209725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs") pod "network-metrics-daemon-k9ljx" (UID: "af520e56-7ba8-4259-a907-7a5a28c41e5b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:11.484665 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:11.484601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzfw8\" (UniqueName: \"kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8\") pod \"network-check-target-qsxfk\" (UID: \"9b236333-c490-4f99-b91f-d44b01461c72\") " pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:11.484803 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:11.484766 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:02:11.484803 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:11.484791 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:02:11.484803 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:11.484805 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lzfw8 for pod openshift-network-diagnostics/network-check-target-qsxfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:11.484918 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:11.484866 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8 podName:9b236333-c490-4f99-b91f-d44b01461c72 nodeName:}" failed. No retries permitted until 2026-07-07 22:02:12.484846632 +0000 UTC m=+3.074225310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lzfw8" (UniqueName: "kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8") pod "network-check-target-qsxfk" (UID: "9b236333-c490-4f99-b91f-d44b01461c72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:12.585415 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:12.585109 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:02:12.684942 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:12.684877 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:02:12.841063 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:12.841026 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 21:57:11 +0000 UTC" deadline="2028-02-20 20:30:59.683205031 +0000 UTC" Jul 07 22:02:12.841063 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:12.841059 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14230h28m46.842149967s" Jul 07 22:02:12.964095 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:12.964073 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:12.964247 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:12.964200 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:12.964720 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:12.964699 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:12.964818 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:12.964797 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:12.981686 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:12.981624 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" event={"ID":"29d5e9a5-3040-4190-af5b-d9e97c3077ca","Type":"ContainerStarted","Data":"743dc63b60117b466e09c9221e551e66c17f0b6dcde6355086221f01aa52eb45"} Jul 07 22:02:12.992323 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:12.992295 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2hj4g" event={"ID":"5c272d87-a60e-4d9f-b942-655bbcb9b788","Type":"ContainerStarted","Data":"1b3e2c2d5aa2cff1b2869f0861595f6925af3fefe7ff3b52876c2af8569fa209"} Jul 07 22:02:13.008327 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.008289 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" event={"ID":"43af0cd5-5dfe-45f2-a50e-84bcda42d705","Type":"ContainerStarted","Data":"a3b2efe3a4b8a69bd8474c843289996e515667e7ecfc81c4350e66d35f8b9b34"} Jul 07 22:02:13.026901 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.026857 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7sxcg" event={"ID":"c41ea711-d80d-4533-a02f-5328426cedb9","Type":"ContainerStarted","Data":"52b1189edd1720ec49519141fb50bd51d789bd2839ec245dc0fdcfd11b01dbbe"} Jul 07 22:02:13.040638 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.040610 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hfthf" event={"ID":"eaf8d762-016e-43f4-bcce-ffd737a1641a","Type":"ContainerStarted","Data":"507cc4384b3284c27902677c252abb22a2cd037fd768477aebcd596f03345de2"} Jul 07 22:02:13.051888 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.051858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-68.ec2.internal" event={"ID":"2cc3ad308bf9860b5b3bfada611aed6f","Type":"ContainerStarted","Data":"9477d825c59ad9be7cf7e7195ebe02e8c17ec66cf1f30a86fb856185e8595fe6"} Jul 07 22:02:13.064557 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.063809 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" event={"ID":"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc","Type":"ContainerStarted","Data":"1015a2f84e5823640cfcba721470b1455a6ab4e11599e8b6a284ddb5e0a04d12"} Jul 07 22:02:13.076297 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.076272 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-glszg" event={"ID":"0fc51f9e-2d89-4649-8992-87f23e1e7721","Type":"ContainerStarted","Data":"ab156bd5017fd238121f25ef9aeb3f4c587b63068524b976983369e93df116cd"} Jul 07 22:02:13.091881 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.089319 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" event={"ID":"d4aebacaa3d93a6d5bf1c346f5aa8f73","Type":"ContainerStarted","Data":"4a1010a2aa45ec2105d78b453fe42ba070f5d0b30fa264149d82ffe7bef721b7"} Jul 07 22:02:13.095365 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:13.095290 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" podUID="bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc" Jul 07 22:02:13.100965 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.100914 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zd4nf" event={"ID":"7285ba25-8291-4c0a-acd4-b986c4652f41","Type":"ContainerStarted","Data":"4007992b87a16cfe33729ef9b362c44bb2fa43f8ed8c2b90476864162fa58166"} Jul 07 22:02:13.112587 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.112557 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wq9vl" event={"ID":"ae3c8757-074f-43f8-9f32-5a69e78843b9","Type":"ContainerStarted","Data":"222bf4d301fb7f711daa376bebecd7d80d68ef5df21cc2f14250d6b7c2b8405d"} Jul 07 22:02:13.417414 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.417346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:13.417626 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.417434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:13.417626 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:13.417576 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:13.417747 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:13.417635 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret podName:60d1d68e-fcca-4c49-9d19-91608847c10b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:15.417616804 +0000 UTC m=+4.981715214 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret") pod "global-pull-secret-syncer-59xcs" (UID: "60d1d68e-fcca-4c49-9d19-91608847c10b") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:13.418023 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:13.418002 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:13.418082 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:13.418069 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs podName:af520e56-7ba8-4259-a907-7a5a28c41e5b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:15.418046456 +0000 UTC m=+4.982144861 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs") pod "network-metrics-daemon-k9ljx" (UID: "af520e56-7ba8-4259-a907-7a5a28c41e5b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:13.518604 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.518563 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzfw8\" (UniqueName: \"kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8\") pod \"network-check-target-qsxfk\" (UID: \"9b236333-c490-4f99-b91f-d44b01461c72\") " pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:13.518816 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:13.518797 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:02:13.518895 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:13.518836 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:02:13.518895 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:13.518851 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lzfw8 for pod openshift-network-diagnostics/network-check-target-qsxfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:13.518996 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:13.518913 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8 podName:9b236333-c490-4f99-b91f-d44b01461c72 nodeName:}" failed. No retries permitted until 2026-07-07 22:02:15.518894546 +0000 UTC m=+5.082992956 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lzfw8" (UniqueName: "kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8") pod "network-check-target-qsxfk" (UID: "9b236333-c490-4f99-b91f-d44b01461c72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:13.842100 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.842035 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 21:57:11 +0000 UTC" deadline="2027-12-17 04:51:46.345972241 +0000 UTC" Jul 07 22:02:13.842100 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.842099 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12654h49m32.503877652s" Jul 07 22:02:13.954186 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:13.954155 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:13.954366 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:13.954312 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:14.532033 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:14.532000 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:02:14.954253 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:14.954217 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:14.954738 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:14.954363 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:14.956812 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:14.956554 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:14.956812 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:14.956693 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:15.434434 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:15.434394 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:15.434634 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:15.434483 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:15.434717 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:15.434636 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:15.434717 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:15.434637 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:15.434717 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:15.434707 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret podName:60d1d68e-fcca-4c49-9d19-91608847c10b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:19.434688441 +0000 UTC m=+8.998786849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret") pod "global-pull-secret-syncer-59xcs" (UID: "60d1d68e-fcca-4c49-9d19-91608847c10b") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:15.434867 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:15.434728 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs podName:af520e56-7ba8-4259-a907-7a5a28c41e5b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:19.43471797 +0000 UTC m=+8.998816377 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs") pod "network-metrics-daemon-k9ljx" (UID: "af520e56-7ba8-4259-a907-7a5a28c41e5b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:15.535912 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:15.535873 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzfw8\" (UniqueName: \"kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8\") pod \"network-check-target-qsxfk\" (UID: \"9b236333-c490-4f99-b91f-d44b01461c72\") " pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:15.536576 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:15.536091 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:02:15.536576 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:15.536117 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:02:15.536576 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:15.536130 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lzfw8 for pod openshift-network-diagnostics/network-check-target-qsxfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:15.536576 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:15.536187 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8 podName:9b236333-c490-4f99-b91f-d44b01461c72 nodeName:}" failed. No retries permitted until 2026-07-07 22:02:19.536169612 +0000 UTC m=+9.100268033 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lzfw8" (UniqueName: "kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8") pod "network-check-target-qsxfk" (UID: "9b236333-c490-4f99-b91f-d44b01461c72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:15.954144 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:15.953657 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:15.954144 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:15.953797 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:16.954102 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:16.954067 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:16.954279 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:16.954106 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:16.954279 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:16.954205 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:16.954692 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:16.954345 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:17.954154 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:17.954121 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:17.954324 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:17.954235 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:18.954170 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:18.954139 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:18.954658 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:18.954173 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:18.954658 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:18.954272 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:18.954658 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:18.954364 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:19.466759 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:19.466730 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:19.466928 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:19.466798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:19.466989 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:19.466950 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:19.467033 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:19.467011 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret podName:60d1d68e-fcca-4c49-9d19-91608847c10b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:27.466993517 +0000 UTC m=+17.031091924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret") pod "global-pull-secret-syncer-59xcs" (UID: "60d1d68e-fcca-4c49-9d19-91608847c10b") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:19.467361 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:19.467338 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:19.467479 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:19.467385 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs podName:af520e56-7ba8-4259-a907-7a5a28c41e5b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:27.467373519 +0000 UTC m=+17.031471923 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs") pod "network-metrics-daemon-k9ljx" (UID: "af520e56-7ba8-4259-a907-7a5a28c41e5b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:19.567460 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:19.567427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzfw8\" (UniqueName: \"kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8\") pod \"network-check-target-qsxfk\" (UID: \"9b236333-c490-4f99-b91f-d44b01461c72\") " pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:19.567656 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:19.567594 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:02:19.567656 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:19.567616 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:02:19.567656 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:19.567627 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lzfw8 for pod openshift-network-diagnostics/network-check-target-qsxfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:19.567790 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:19.567675 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8 podName:9b236333-c490-4f99-b91f-d44b01461c72 nodeName:}" failed. No retries permitted until 2026-07-07 22:02:27.567661684 +0000 UTC m=+17.131760088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lzfw8" (UniqueName: "kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8") pod "network-check-target-qsxfk" (UID: "9b236333-c490-4f99-b91f-d44b01461c72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:19.953998 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:19.953968 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:19.954177 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:19.954107 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:20.954906 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:20.954400 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:20.954906 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:20.954538 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:20.954906 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:20.954575 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:20.954906 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:20.954684 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:21.954721 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:21.954470 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:21.954856 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:21.954765 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:22.135050 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.135013 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hfthf" event={"ID":"eaf8d762-016e-43f4-bcce-ffd737a1641a","Type":"ContainerStarted","Data":"b3f8125e32bdf6951397aedc92e950ce24130fc958d811c043be5cb045d12632"} Jul 07 22:02:22.136834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.136807 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-68.ec2.internal" event={"ID":"2cc3ad308bf9860b5b3bfada611aed6f","Type":"ContainerStarted","Data":"6a1d4e4b6e3b9141eb1cce446982ede8447006fd47d3ddb5c6e0d51b3ccc3f7e"} Jul 07 22:02:22.138551 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.138508 2580 generic.go:356] "Generic (PLEG): container finished" podID="d4aebacaa3d93a6d5bf1c346f5aa8f73" containerID="d8865d1578e0de21496b41a9e42db3d9d6cf5d2ed3942557536cab94e880842c" exitCode=0 Jul 07 22:02:22.138672 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.138622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" event={"ID":"d4aebacaa3d93a6d5bf1c346f5aa8f73","Type":"ContainerDied","Data":"d8865d1578e0de21496b41a9e42db3d9d6cf5d2ed3942557536cab94e880842c"} Jul 07 22:02:22.143079 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.143059 2580 generic.go:356] "Generic (PLEG): container finished" podID="7285ba25-8291-4c0a-acd4-b986c4652f41" containerID="d8a53af0da57e0435523e795f653ec5e8779d05818692ecc73bcbeda65bba241" exitCode=0 Jul 07 22:02:22.143178 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.143140 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zd4nf" event={"ID":"7285ba25-8291-4c0a-acd4-b986c4652f41","Type":"ContainerDied","Data":"d8a53af0da57e0435523e795f653ec5e8779d05818692ecc73bcbeda65bba241"} Jul 07 22:02:22.144666 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.144592 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" event={"ID":"29d5e9a5-3040-4190-af5b-d9e97c3077ca","Type":"ContainerStarted","Data":"b479ddd2710e46cef4f72bb9adafdb9fb372932bc731252a7bbc17737a649edd"} Jul 07 22:02:22.146155 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.146137 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2hj4g" event={"ID":"5c272d87-a60e-4d9f-b942-655bbcb9b788","Type":"ContainerStarted","Data":"a5def811cfa5c33097407c88afe8d1fcd0149e3153a0646ec229b1a8a25d536a"} Jul 07 22:02:22.147870 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.147842 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" event={"ID":"43af0cd5-5dfe-45f2-a50e-84bcda42d705","Type":"ContainerStarted","Data":"303a6a9f77bc054b505b8421c9dc8e76f344d333b013c3eef4beed2bd70e2444"} Jul 07 22:02:22.149247 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.149227 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7sxcg" event={"ID":"c41ea711-d80d-4533-a02f-5328426cedb9","Type":"ContainerStarted","Data":"51ff4b5d4b0aadf16a9130ee9c859fd88d7b93bed6e9cddd3a660aeef757e65e"} Jul 07 22:02:22.151824 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.151778 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hfthf" podStartSLOduration=2.875180548 podStartE2EDuration="12.151761201s" totalImagesPullingTime="9.276580653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:02:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:02:22.150827637 +0000 UTC m=+11.714926069" watchObservedRunningTime="2026-07-07 22:02:22.151761201 +0000 UTC m=+11.715859628" Jul 07 22:02:22.197079 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.197026 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-68.ec2.internal" podStartSLOduration=11.197008999 podStartE2EDuration="11.197008999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:02:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:02:22.196693325 +0000 UTC m=+11.760791753" watchObservedRunningTime="2026-07-07 22:02:22.197008999 +0000 UTC m=+11.761107424" Jul 07 22:02:22.212077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.212028 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b4fxx" podStartSLOduration=2.9454129719999997 podStartE2EDuration="12.212010014s" totalImagesPullingTime="9.266597042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:02:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:02:22.21168897 +0000 UTC m=+11.775787397" watchObservedRunningTime="2026-07-07 22:02:22.212010014 +0000 UTC m=+11.776108443" Jul 07 22:02:22.226001 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.225925 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2hj4g" podStartSLOduration=2.008004534 podStartE2EDuration="11.225905853s" totalImagesPullingTime="9.217901319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:02:22.225881356 +0000 UTC m=+11.789979785" watchObservedRunningTime="2026-07-07 22:02:22.225905853 +0000 UTC m=+11.790004313" Jul 07 22:02:22.689686 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.689661 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 22:02:22.856355 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.856216 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T22:02:22.689700332Z","UUID":"4a7960ce-6496-49bc-b802-abec6dd5ab7a","Handler":null,"Name":"","Endpoint":""} Jul 07 22:02:22.857711 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.857692 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 22:02:22.857815 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.857727 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 22:02:22.953762 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.953735 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:22.953907 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:22.953834 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:22.953954 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:22.953936 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:22.954110 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:22.954086 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:23.153679 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:23.153591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-glszg" event={"ID":"0fc51f9e-2d89-4649-8992-87f23e1e7721","Type":"ContainerStarted","Data":"49e258f2fd4c80655a61bb813701bbfc64d25040f6f5c40ac0d6c1e3f2c3a7e3"} Jul 07 22:02:23.156295 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:23.156261 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" event={"ID":"d4aebacaa3d93a6d5bf1c346f5aa8f73","Type":"ContainerStarted","Data":"fe60599d8af5379efc5ece5e2677523bab31e52eaa1170bd7c3804bcef8b1d8e"} Jul 07 22:02:23.158733 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:23.158702 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" event={"ID":"29d5e9a5-3040-4190-af5b-d9e97c3077ca","Type":"ContainerStarted","Data":"3ee404f575e6443cf58c832e0aca490db095beb626f7b8d8280811ecf8d5e053"} Jul 07 22:02:23.170881 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:23.170834 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7sxcg" podStartSLOduration=3.910696481 podStartE2EDuration="13.170816479s" totalImagesPullingTime="9.260119998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:02:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:02:22.241496588 +0000 UTC m=+11.805595015" watchObservedRunningTime="2026-07-07 22:02:23.170816479 +0000 UTC m=+12.734914906" Jul 07 22:02:23.171148 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:23.171118 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-glszg" podStartSLOduration=2.97957629 podStartE2EDuration="12.171107088s" totalImagesPullingTime="9.191530798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:02:23.170549636 +0000 UTC m=+12.734648064" watchObservedRunningTime="2026-07-07 22:02:23.171107088 +0000 UTC m=+12.735205514" Jul 07 22:02:23.184573 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:23.184509 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-68.ec2.internal" podStartSLOduration=12.184496004 podStartE2EDuration="12.184496004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:02:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:02:23.184298555 +0000 UTC m=+12.748396982" watchObservedRunningTime="2026-07-07 22:02:23.184496004 +0000 UTC m=+12.748594432" Jul 07 22:02:23.954676 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:23.954596 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:23.954838 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:23.954744 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:24.163634 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:24.163595 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" event={"ID":"29d5e9a5-3040-4190-af5b-d9e97c3077ca","Type":"ContainerStarted","Data":"cb02f90f58c7eec8b246f8984dddaa2237dd8853a2e14e4688033105e11255f8"} Jul 07 22:02:24.181545 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:24.181476 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jt65" podStartSLOduration=2.164417991 podStartE2EDuration="13.181457052s" totalImagesPullingTime="11.017039061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:02:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:02:24.181000042 +0000 UTC m=+13.745098462" watchObservedRunningTime="2026-07-07 22:02:24.181457052 +0000 UTC m=+13.745555479" Jul 07 22:02:24.953737 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:24.953702 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:24.953914 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:24.953711 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:24.953914 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:24.953809 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:24.953914 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:24.953898 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:25.954400 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:25.954189 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:25.954938 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:25.954500 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:26.867262 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:26.867225 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:26.868209 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:26.868185 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:26.954573 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:26.954511 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:26.955015 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:26.954658 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:26.955094 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:26.955008 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:26.955148 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:26.955113 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:27.526440 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:27.526404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:27.526663 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:27.526501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:27.526663 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:27.526584 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:27.526663 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:27.526635 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret podName:60d1d68e-fcca-4c49-9d19-91608847c10b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:43.526621176 +0000 UTC m=+33.090719580 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret") pod "global-pull-secret-syncer-59xcs" (UID: "60d1d68e-fcca-4c49-9d19-91608847c10b") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:27.526663 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:27.526644 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:27.526874 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:27.526705 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs podName:af520e56-7ba8-4259-a907-7a5a28c41e5b nodeName:}" failed. No retries permitted until 2026-07-07 22:02:43.526688204 +0000 UTC m=+33.090786615 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs") pod "network-metrics-daemon-k9ljx" (UID: "af520e56-7ba8-4259-a907-7a5a28c41e5b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:27.627752 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:27.627716 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzfw8\" (UniqueName: \"kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8\") pod \"network-check-target-qsxfk\" (UID: \"9b236333-c490-4f99-b91f-d44b01461c72\") " pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:27.627915 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:27.627899 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:02:27.627994 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:27.627921 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:02:27.627994 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:27.627932 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lzfw8 for pod openshift-network-diagnostics/network-check-target-qsxfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:27.627994 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:27.627986 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8 podName:9b236333-c490-4f99-b91f-d44b01461c72 nodeName:}" failed. No retries permitted until 2026-07-07 22:02:43.627967092 +0000 UTC m=+33.192065515 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lzfw8" (UniqueName: "kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8") pod "network-check-target-qsxfk" (UID: "9b236333-c490-4f99-b91f-d44b01461c72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:27.953852 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:27.953818 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:27.954064 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:27.953948 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:28.953618 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:28.953579 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:28.954076 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:28.953708 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:28.954145 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:28.954120 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:28.954267 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:28.954233 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:29.953588 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:29.953554 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:29.953773 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:29.953694 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:30.955793 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:30.955085 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:30.955793 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:30.955187 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:30.955793 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:30.955486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:30.955793 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:30.955607 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:31.182478 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:31.182395 2580 generic.go:356] "Generic (PLEG): container finished" podID="7285ba25-8291-4c0a-acd4-b986c4652f41" containerID="b41ba2d4f29bd60a813ddd21633ef4cda1a24dc353f4a1c37618353e692f5b31" exitCode=0 Jul 07 22:02:31.182654 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:31.182484 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zd4nf" event={"ID":"7285ba25-8291-4c0a-acd4-b986c4652f41","Type":"ContainerDied","Data":"b41ba2d4f29bd60a813ddd21633ef4cda1a24dc353f4a1c37618353e692f5b31"} Jul 07 22:02:31.186906 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:31.186852 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wq9vl" event={"ID":"ae3c8757-074f-43f8-9f32-5a69e78843b9","Type":"ContainerStarted","Data":"d0a800d35ce57a98c0ae53d4bdfdfef7ea0f16e1c1285d5d6118ca209387f82c"} Jul 07 22:02:31.226934 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:31.226889 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wq9vl" podStartSLOduration=1.488092546 podStartE2EDuration="20.226867972s" totalImagesPullingTime="18.738775426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:02:31.226570849 +0000 UTC m=+20.790669265" watchObservedRunningTime="2026-07-07 22:02:31.226867972 +0000 UTC m=+20.790966462" Jul 07 22:02:31.393104 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:31.393070 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:31.483930 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:31.483842 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:31.953708 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:31.953658 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:31.953891 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:31.953795 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:32.609242 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:32.609208 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:32.954135 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:32.954109 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:32.954263 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:32.954245 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:32.954592 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:32.954567 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:32.954693 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:32.954675 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:33.193931 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:33.193846 2580 generic.go:356] "Generic (PLEG): container finished" podID="7285ba25-8291-4c0a-acd4-b986c4652f41" containerID="c7aaba4c39b6779f2462b2fde6dc88d95295e0a4e01fd4aaac3090a8f0892122" exitCode=0 Jul 07 22:02:33.193931 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:33.193914 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zd4nf" event={"ID":"7285ba25-8291-4c0a-acd4-b986c4652f41","Type":"ContainerDied","Data":"c7aaba4c39b6779f2462b2fde6dc88d95295e0a4e01fd4aaac3090a8f0892122"} Jul 07 22:02:33.954004 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:33.953966 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:33.954473 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:33.954133 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:34.814673 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:34.814641 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:34.814926 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:34.814776 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 22:02:34.815454 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:34.815371 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2hj4g" Jul 07 22:02:34.879151 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:34.879119 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:34.954528 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:34.954476 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:34.954995 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:34.954636 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:34.954995 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:34.954735 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:34.954995 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:34.954818 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:35.804146 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:35.804106 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:35.905389 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:35.905335 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:35.954529 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:35.954478 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:35.954709 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:35.954633 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:36.953930 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:36.953897 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:36.954124 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:36.954006 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:36.954124 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:36.954048 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:36.954217 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:36.954126 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:37.953683 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:37.953651 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:37.954044 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:37.953750 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:38.953938 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:38.953717 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:38.954652 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:38.953716 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:38.954652 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:38.954050 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:38.954987 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:38.954947 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:39.209166 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:39.209137 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/ovn-acl-logging/0.log" Jul 07 22:02:39.209466 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:39.209446 2580 generic.go:356] "Generic (PLEG): container finished" podID="bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc" containerID="851a6bc1b6bad349a9f5c5132f9ad7b669ae101bd3856d2ba36593cb93702d24" exitCode=1 Jul 07 22:02:39.209559 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:39.209536 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" event={"ID":"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc","Type":"ContainerStarted","Data":"48cfcd7d96decc706b5fba491351293a91e9903f29e5f9f6416ccfad3d13a39b"} Jul 07 22:02:39.209623 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:39.209609 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" event={"ID":"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc","Type":"ContainerStarted","Data":"71e89cfed33f6fcd70f78480903ddf8551161f91c540ddec144dc0f4f6d31ffb"} Jul 07 22:02:39.209664 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:39.209652 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" event={"ID":"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc","Type":"ContainerStarted","Data":"c698eb1b130c6deae6f33a37a7daff533cf7d774f93fa9821853e05b1355100b"} Jul 07 22:02:39.209705 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:39.209680 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" event={"ID":"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc","Type":"ContainerStarted","Data":"e2af27378955b87ba83863863984724be4eff4f8cf36b99db25265b3413c63f6"} Jul 07 22:02:39.209735 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:39.209712 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" event={"ID":"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc","Type":"ContainerDied","Data":"851a6bc1b6bad349a9f5c5132f9ad7b669ae101bd3856d2ba36593cb93702d24"} Jul 07 22:02:39.209773 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:39.209760 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" event={"ID":"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc","Type":"ContainerStarted","Data":"e69cd2f1c35d5fe573c75b31b4729b814af3062c93d2230b885541ddb8b8386c"} Jul 07 22:02:39.211708 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:39.211682 2580 generic.go:356] "Generic (PLEG): container finished" podID="7285ba25-8291-4c0a-acd4-b986c4652f41" containerID="80bff7495cc05674f8ee89ab9112f8a37be5214880276ba0ffc5e0eee3ff97de" exitCode=0 Jul 07 22:02:39.211810 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:39.211725 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zd4nf" event={"ID":"7285ba25-8291-4c0a-acd4-b986c4652f41","Type":"ContainerDied","Data":"80bff7495cc05674f8ee89ab9112f8a37be5214880276ba0ffc5e0eee3ff97de"} Jul 07 22:02:39.954080 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:39.954046 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:39.954539 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:39.954164 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:40.955244 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:40.955146 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:40.955737 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:40.955267 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:40.955737 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:40.955317 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:40.955737 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:40.955417 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:41.219728 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:41.219657 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/ovn-acl-logging/0.log" Jul 07 22:02:41.220029 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:41.219988 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" event={"ID":"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc","Type":"ContainerStarted","Data":"9e7f6eca9e65ca24aa0af272948e7d7c7597271043784a559135bfff32c48986"} Jul 07 22:02:41.509741 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:41.509655 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:41.607087 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:41.607047 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:41.938133 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:41.938097 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:41.954146 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:41.954120 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:41.954283 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:41.954233 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:42.001642 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:42.001612 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:42.954029 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:42.953984 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:42.954029 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:42.954006 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:42.954265 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:42.954129 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:42.954327 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:42.954260 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:43.397273 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:43.397137 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:43.519709 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:43.519620 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:43.547064 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:43.547039 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:43.547201 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:43.547085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:43.547201 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:43.547176 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:43.547201 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:43.547191 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:43.547327 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:43.547223 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret podName:60d1d68e-fcca-4c49-9d19-91608847c10b nodeName:}" failed. No retries permitted until 2026-07-07 22:03:15.547211987 +0000 UTC m=+65.111310392 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret") pod "global-pull-secret-syncer-59xcs" (UID: "60d1d68e-fcca-4c49-9d19-91608847c10b") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:02:43.547327 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:43.547252 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs podName:af520e56-7ba8-4259-a907-7a5a28c41e5b nodeName:}" failed. No retries permitted until 2026-07-07 22:03:15.547232859 +0000 UTC m=+65.111331280 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs") pod "network-metrics-daemon-k9ljx" (UID: "af520e56-7ba8-4259-a907-7a5a28c41e5b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:02:43.603642 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:43.603621 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:43.647547 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:43.647524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzfw8\" (UniqueName: \"kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8\") pod \"network-check-target-qsxfk\" (UID: \"9b236333-c490-4f99-b91f-d44b01461c72\") " pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:43.647653 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:43.647639 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:02:43.647689 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:43.647660 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:02:43.647689 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:43.647669 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lzfw8 for pod openshift-network-diagnostics/network-check-target-qsxfk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:43.647764 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:43.647709 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8 podName:9b236333-c490-4f99-b91f-d44b01461c72 nodeName:}" failed. No retries permitted until 2026-07-07 22:03:15.647697909 +0000 UTC m=+65.211796313 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lzfw8" (UniqueName: "kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8") pod "network-check-target-qsxfk" (UID: "9b236333-c490-4f99-b91f-d44b01461c72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:02:43.749612 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:43.749564 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:43.847772 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:43.847747 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:43.931935 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:43.931907 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:43.953688 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:43.953666 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:43.953786 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:43.953758 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:44.094897 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.094868 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:44.199872 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.199807 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:44.229526 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.229492 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/ovn-acl-logging/0.log" Jul 07 22:02:44.229924 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.229900 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" event={"ID":"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc","Type":"ContainerStarted","Data":"d327e0380490027b95fd7bcf9f8c05be26ee7153510ff239574cc62665cc5ae0"} Jul 07 22:02:44.230149 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.230133 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:44.230406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.230292 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:44.230406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.230356 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:44.230406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.230301 2580 scope.go:117] "RemoveContainer" containerID="851a6bc1b6bad349a9f5c5132f9ad7b669ae101bd3856d2ba36593cb93702d24" Jul 07 22:02:44.235569 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.235473 2580 generic.go:356] "Generic (PLEG): container finished" podID="7285ba25-8291-4c0a-acd4-b986c4652f41" containerID="b94eb355babcddc98134a57dcaecd07802f03d3f71dc6a4070844574205f7636" exitCode=0 Jul 07 22:02:44.235569 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.235533 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zd4nf" event={"ID":"7285ba25-8291-4c0a-acd4-b986c4652f41","Type":"ContainerDied","Data":"b94eb355babcddc98134a57dcaecd07802f03d3f71dc6a4070844574205f7636"} Jul 07 22:02:44.248173 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.248144 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:44.249762 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.249740 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:02:44.794228 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.794194 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:44.861841 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.861814 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:44.930746 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.930720 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hfthf_eaf8d762-016e-43f4-bcce-ffd737a1641a/dns-node-resolver/0.log" Jul 07 22:02:44.954036 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.954009 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:44.954157 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:44.954129 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:44.954228 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:44.954211 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:44.954331 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:44.954313 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:45.054114 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.053927 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:45.153725 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.153697 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:45.230566 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.230506 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-59xcs"] Jul 07 22:02:45.231127 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.231106 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k9ljx"] Jul 07 22:02:45.231255 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.231239 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:45.231427 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:45.231403 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:45.231950 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.231927 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qsxfk"] Jul 07 22:02:45.241874 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.241849 2580 generic.go:356] "Generic (PLEG): container finished" podID="7285ba25-8291-4c0a-acd4-b986c4652f41" containerID="d917c3b82f6a2839a4c4504b1716cb54955e87b9317d1a06151b02b7bdabe1ca" exitCode=0 Jul 07 22:02:45.241962 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.241920 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zd4nf" event={"ID":"7285ba25-8291-4c0a-acd4-b986c4652f41","Type":"ContainerDied","Data":"d917c3b82f6a2839a4c4504b1716cb54955e87b9317d1a06151b02b7bdabe1ca"} Jul 07 22:02:45.246209 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.246183 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/ovn-acl-logging/0.log" Jul 07 22:02:45.246693 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.246637 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" event={"ID":"bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc","Type":"ContainerStarted","Data":"79a5527f6650e2990500289ef03dedcccf2a437f354e9fade6af36a55f7e395e"} Jul 07 22:02:45.246763 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.246660 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:45.246839 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.246682 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:45.246869 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:45.246852 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:45.246929 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:45.246917 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:45.288884 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.288825 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:45.293304 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.293264 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" podStartSLOduration=8.110259657 podStartE2EDuration="34.293250924s" totalImagesPullingTime="26.182991267s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=2 observedRunningTime="2026-07-07 22:02:45.292438556 +0000 UTC m=+34.856536993" watchObservedRunningTime="2026-07-07 22:02:45.293250924 +0000 UTC m=+34.857349350" Jul 07 22:02:45.348670 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.348649 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:45.679093 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.679055 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:45.728829 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.728810 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7sxcg_c41ea711-d80d-4533-a02f-5328426cedb9/node-ca/0.log" Jul 07 22:02:45.736985 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:45.736948 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:46.252040 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:46.252009 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zd4nf" event={"ID":"7285ba25-8291-4c0a-acd4-b986c4652f41","Type":"ContainerStarted","Data":"ba9fd23658e007c9c8ccc5c99ac484a4e05bbcb48a304e6d7ee3058e2eb7fcc3"} Jul 07 22:02:46.273978 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:46.273936 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zd4nf" podStartSLOduration=6.243010559 podStartE2EDuration="35.273922708s" totalImagesPullingTime="29.030912149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:02:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:02:46.272867687 +0000 UTC m=+35.836966110" watchObservedRunningTime="2026-07-07 22:02:46.273922708 +0000 UTC m=+35.838021148" Jul 07 22:02:46.399949 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:46.399920 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:46.458814 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:46.458781 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:46.954454 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:46.954423 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:46.954672 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:46.954424 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:46.954672 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:46.954569 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:46.954672 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:46.954568 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:46.954672 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:46.954643 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:46.954890 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:46.954786 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:47.305641 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:47.305615 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:47.410026 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:47.410001 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:48.953834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:48.953802 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:48.953834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:48.953814 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:48.953834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:48.953830 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:48.954413 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:48.953890 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:48.954413 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:48.953960 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:48.954413 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:48.954009 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:50.402290 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:50.402257 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:50.954487 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:50.954459 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:50.954647 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:50.954556 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:50.954647 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:50.954626 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:50.954764 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:50.954714 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:50.954801 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:50.954764 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:50.954801 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:50.954764 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:51.916808 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:51.916759 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:52.112339 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:52.112272 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:52.215002 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:52.214931 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:52.313807 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:52.313778 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:52.414825 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:52.414797 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:52.600706 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:52.600678 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:52.954255 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:52.954164 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:52.954726 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:52.954172 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:52.954726 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:52.954298 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:52.954726 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:52.954172 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:52.954726 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:52.954343 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:52.954726 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:52.954405 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:53.817891 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:53.817860 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:53.875182 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:53.875150 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:54.003626 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:54.003587 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:54.202874 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:54.202837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:54.407599 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:54.407572 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:54.623657 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:54.623622 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:54.954006 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:54.953934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:54.954179 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:54.954040 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:54.954179 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:54.954046 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:54.954179 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:54.954108 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:54.954179 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:54.954130 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:54.954308 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:54.954231 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:55.168558 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:55.168346 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:55.266233 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:55.266160 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:55.405984 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:55.405953 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:55.467776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:55.467747 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:55.613818 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:55.613790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:56.001067 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:56.000989 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:56.242767 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:56.242732 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:56.315179 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:56.315104 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:56.409261 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:56.409231 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:56.806915 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:56.806887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:56.953640 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:56.953615 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:56.953640 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:56.953649 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:56.953892 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:56.953620 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:56.953892 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:56.953735 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:56.953892 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:56.953798 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:56.953892 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:56.953875 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:57.408645 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:57.408614 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:57.804749 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:57.804663 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:58.225115 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:58.225085 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:58.328734 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:58.328700 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:58.425633 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:58.425602 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:58.954032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:58.953996 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:02:58.954235 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:58.953994 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:02:58.954235 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:58.954098 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:02:58.954235 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:58.954003 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:02:58.954235 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:58.954180 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:02:58.954453 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:02:58.954240 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:02:59.008376 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:59.008340 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:59.406950 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:59.406919 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:02:59.603456 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:02:59.603427 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:00.212228 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:00.212194 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:00.311530 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:00.311479 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:00.413241 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:00.413205 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:00.606723 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:00.606695 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:00.954897 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:00.954814 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:03:00.955040 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:00.954927 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:03:00.955040 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:00.955003 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:03:00.955146 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:00.955088 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:03:00.955146 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:00.955111 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:03:00.955248 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:00.955145 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:03:01.010253 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:01.010225 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:01.222750 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:01.222672 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:01.608728 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:01.608695 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:02.207804 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:02.207773 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:02.403159 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:02.403131 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:02.603354 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:02.603328 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:02.953654 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:02.953576 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:03:02.953999 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:02.953584 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:03:02.953999 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:02.953689 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:03:02.953999 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:02.953687 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:03:02.953999 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:02.953779 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:03:02.953999 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:02.953852 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:03:03.954981 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:03.954947 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:04.027257 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:04.027220 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:04.082654 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:04.082622 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:04.244662 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:04.244566 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:04.433062 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:04.432999 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:04.625674 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:04.625639 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:04.726570 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:04.726535 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:04.954649 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:04.954575 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:03:04.954813 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:04.954574 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:03:04.954813 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:04.954681 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsxfk" podUID="9b236333-c490-4f99-b91f-d44b01461c72" Jul 07 22:03:04.954813 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:04.954584 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:03:04.954813 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:04.954743 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k9ljx" podUID="af520e56-7ba8-4259-a907-7a5a28c41e5b" Jul 07 22:03:04.955029 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:04.954836 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59xcs" podUID="60d1d68e-fcca-4c49-9d19-91608847c10b" Jul 07 22:03:05.025962 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.025934 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:05.127826 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.127792 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:05.284425 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.284346 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:05.399314 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.399265 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:05.499834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.499805 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:05.593479 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.593451 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:05.662256 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.662237 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-68.ec2.internal" event="NodeReady" Jul 07 22:03:05.662383 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.662348 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 22:03:05.664914 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.664894 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:05.708896 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.708860 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vjl8c"] Jul 07 22:03:05.725853 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.725832 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5zv97"] Jul 07 22:03:05.725988 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.725962 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.728169 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.728145 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 22:03:05.728169 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.728163 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 22:03:05.728359 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.728182 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 22:03:05.728359 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.728317 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wwv7h\"" Jul 07 22:03:05.728474 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.728370 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 22:03:05.739837 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.739817 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5zv97"] Jul 07 22:03:05.739924 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.739847 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vjl8c"] Jul 07 22:03:05.739924 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.739917 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5zv97" Jul 07 22:03:05.742372 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.742355 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 22:03:05.742458 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.742418 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lf8bf\"" Jul 07 22:03:05.742728 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.742709 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 22:03:05.742821 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.742793 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 22:03:05.808967 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.808932 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-km8w9"] Jul 07 22:03:05.811092 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.811073 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.811178 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.811102 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88ltg\" (UniqueName: \"kubernetes.io/projected/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-kube-api-access-88ltg\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.811178 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.811123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bq7n\" (UniqueName: \"kubernetes.io/projected/0fdba9d1-084e-419a-badb-52a97c802694-kube-api-access-9bq7n\") pod \"ingress-canary-5zv97\" (UID: \"0fdba9d1-084e-419a-badb-52a97c802694\") " pod="openshift-ingress-canary/ingress-canary-5zv97" Jul 07 22:03:05.811288 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.811245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.811288 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.811275 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0fdba9d1-084e-419a-badb-52a97c802694-cert\") pod \"ingress-canary-5zv97\" (UID: \"0fdba9d1-084e-419a-badb-52a97c802694\") " pod="openshift-ingress-canary/ingress-canary-5zv97" Jul 07 22:03:05.811386 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.811325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-crio-socket\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.811386 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.811360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-data-volume\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.822510 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.822497 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:05.825065 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.825039 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hq4js\"" Jul 07 22:03:05.825184 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.825120 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 22:03:05.825331 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.825314 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 22:03:05.829259 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.829236 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-km8w9"] Jul 07 22:03:05.846940 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.846887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:05.911790 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.911770 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.911907 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.911796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88ltg\" (UniqueName: \"kubernetes.io/projected/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-kube-api-access-88ltg\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.911907 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.911815 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bq7n\" (UniqueName: \"kubernetes.io/projected/0fdba9d1-084e-419a-badb-52a97c802694-kube-api-access-9bq7n\") pod \"ingress-canary-5zv97\" (UID: \"0fdba9d1-084e-419a-badb-52a97c802694\") " pod="openshift-ingress-canary/ingress-canary-5zv97" Jul 07 22:03:05.911907 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.911836 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2fsw\" (UniqueName: \"kubernetes.io/projected/f8b95043-d806-49a7-baeb-451b8d7fac47-kube-api-access-g2fsw\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:05.911907 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.911868 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8b95043-d806-49a7-baeb-451b8d7fac47-metrics-tls\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:05.912106 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.911925 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.912106 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.911941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0fdba9d1-084e-419a-badb-52a97c802694-cert\") pod \"ingress-canary-5zv97\" (UID: \"0fdba9d1-084e-419a-badb-52a97c802694\") " pod="openshift-ingress-canary/ingress-canary-5zv97" Jul 07 22:03:05.912106 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.911955 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8b95043-d806-49a7-baeb-451b8d7fac47-tmp-dir\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:05.912106 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.911989 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-crio-socket\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.912106 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.912012 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f8b95043-d806-49a7-baeb-451b8d7fac47-config-volume\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:05.912106 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.912045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-data-volume\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.912106 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.912090 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-crio-socket\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.912345 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.912325 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-data-volume\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.912478 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.912453 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.915652 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.915633 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.915755 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.915737 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0fdba9d1-084e-419a-badb-52a97c802694-cert\") pod \"ingress-canary-5zv97\" (UID: \"0fdba9d1-084e-419a-badb-52a97c802694\") " pod="openshift-ingress-canary/ingress-canary-5zv97" Jul 07 22:03:05.921007 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.920979 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88ltg\" (UniqueName: \"kubernetes.io/projected/96de9fd5-aa10-4eca-ac08-b8e51cac60b1-kube-api-access-88ltg\") pod \"insights-runtime-extractor-vjl8c\" (UID: \"96de9fd5-aa10-4eca-ac08-b8e51cac60b1\") " pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:05.921131 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:05.921113 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bq7n\" (UniqueName: \"kubernetes.io/projected/0fdba9d1-084e-419a-badb-52a97c802694-kube-api-access-9bq7n\") pod \"ingress-canary-5zv97\" (UID: \"0fdba9d1-084e-419a-badb-52a97c802694\") " pod="openshift-ingress-canary/ingress-canary-5zv97" Jul 07 22:03:06.012878 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.012847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8b95043-d806-49a7-baeb-451b8d7fac47-tmp-dir\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:06.013265 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.012895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f8b95043-d806-49a7-baeb-451b8d7fac47-config-volume\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:06.013265 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.012938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2fsw\" (UniqueName: \"kubernetes.io/projected/f8b95043-d806-49a7-baeb-451b8d7fac47-kube-api-access-g2fsw\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:06.013265 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.012980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8b95043-d806-49a7-baeb-451b8d7fac47-metrics-tls\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:06.013265 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.013227 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8b95043-d806-49a7-baeb-451b8d7fac47-tmp-dir\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:06.013581 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.013560 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f8b95043-d806-49a7-baeb-451b8d7fac47-config-volume\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:06.015339 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.015322 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8b95043-d806-49a7-baeb-451b8d7fac47-metrics-tls\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:06.021455 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.021436 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2fsw\" (UniqueName: \"kubernetes.io/projected/f8b95043-d806-49a7-baeb-451b8d7fac47-kube-api-access-g2fsw\") pod \"dns-default-km8w9\" (UID: \"f8b95043-d806-49a7-baeb-451b8d7fac47\") " pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:06.034740 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.034723 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vjl8c" Jul 07 22:03:06.035714 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.035694 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:06.047605 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.047582 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5zv97" Jul 07 22:03:06.134308 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.134234 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:06.236811 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.236780 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vjl8c"] Jul 07 22:03:06.240300 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:06.240274 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96de9fd5_aa10_4eca_ac08_b8e51cac60b1.slice/crio-7d68208ebfe9eff7a66652821f091f313c2433a7a1c2b8501789a9549dba0326 WatchSource:0}: Error finding container 7d68208ebfe9eff7a66652821f091f313c2433a7a1c2b8501789a9549dba0326: Status 404 returned error can't find the container with id 7d68208ebfe9eff7a66652821f091f313c2433a7a1c2b8501789a9549dba0326 Jul 07 22:03:06.248866 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.248837 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5zv97"] Jul 07 22:03:06.252164 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:06.252137 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fdba9d1_084e_419a_badb_52a97c802694.slice/crio-4f7145d53dd6f8d55b0b3884caa60e2106d51892784481fdb404a1346d7b1c54 WatchSource:0}: Error finding container 4f7145d53dd6f8d55b0b3884caa60e2106d51892784481fdb404a1346d7b1c54: Status 404 returned error can't find the container with id 4f7145d53dd6f8d55b0b3884caa60e2106d51892784481fdb404a1346d7b1c54 Jul 07 22:03:06.296886 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.296849 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5zv97" event={"ID":"0fdba9d1-084e-419a-badb-52a97c802694","Type":"ContainerStarted","Data":"4f7145d53dd6f8d55b0b3884caa60e2106d51892784481fdb404a1346d7b1c54"} Jul 07 22:03:06.297406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.297382 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-km8w9"] Jul 07 22:03:06.298104 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.298078 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vjl8c" event={"ID":"96de9fd5-aa10-4eca-ac08-b8e51cac60b1","Type":"ContainerStarted","Data":"7d68208ebfe9eff7a66652821f091f313c2433a7a1c2b8501789a9549dba0326"} Jul 07 22:03:06.300561 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:06.300539 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8b95043_d806_49a7_baeb_451b8d7fac47.slice/crio-1797115074d2ab382db7e919e7d2db1f6320d9c87cb7c779299bf0320e093299 WatchSource:0}: Error finding container 1797115074d2ab382db7e919e7d2db1f6320d9c87cb7c779299bf0320e093299: Status 404 returned error can't find the container with id 1797115074d2ab382db7e919e7d2db1f6320d9c87cb7c779299bf0320e093299 Jul 07 22:03:06.314946 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.314925 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:06.377011 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.376983 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:06.475576 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.475494 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:06.706586 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.706549 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:06.810921 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.810843 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:06.910726 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.910693 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:06.953764 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.953732 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:03:06.954563 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.954546 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:03:06.955270 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.955215 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:03:06.956569 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.956268 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 22:03:06.957316 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.957278 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 22:03:06.957533 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.957501 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cx84f\"" Jul 07 22:03:06.958034 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.958014 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 22:03:06.958248 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.958233 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jvn48\"" Jul 07 22:03:06.958443 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:06.958426 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 22:03:07.302482 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:07.302448 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-km8w9" event={"ID":"f8b95043-d806-49a7-baeb-451b8d7fac47","Type":"ContainerStarted","Data":"1797115074d2ab382db7e919e7d2db1f6320d9c87cb7c779299bf0320e093299"} Jul 07 22:03:07.508773 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:07.508681 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:07.898814 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:07.898725 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:07.999677 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:07.999640 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:09.762359 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:09.762327 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:09.814213 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:09.814183 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:10.766266 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:10.766229 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:10.864163 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:10.864125 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:10.929639 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:10.929608 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:10.932372 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:10.932349 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq"] Jul 07 22:03:10.964835 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:10.964806 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq"] Jul 07 22:03:10.964954 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:10.964912 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" Jul 07 22:03:10.967024 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:10.966990 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 22:03:10.967156 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:10.967041 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7j4kt\"" Jul 07 22:03:11.050672 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:11.050608 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a27db392-9562-49ea-a86c-325f3635da98-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-ckzpq\" (UID: \"a27db392-9562-49ea-a86c-325f3635da98\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" Jul 07 22:03:11.151721 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:11.151679 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a27db392-9562-49ea-a86c-325f3635da98-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-ckzpq\" (UID: \"a27db392-9562-49ea-a86c-325f3635da98\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" Jul 07 22:03:11.151878 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:11.151842 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 07 22:03:11.151972 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:11.151959 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a27db392-9562-49ea-a86c-325f3635da98-tls-certificates podName:a27db392-9562-49ea-a86c-325f3635da98 nodeName:}" failed. No retries permitted until 2026-07-07 22:03:11.651931333 +0000 UTC m=+61.216029737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a27db392-9562-49ea-a86c-325f3635da98-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-ckzpq" (UID: "a27db392-9562-49ea-a86c-325f3635da98") : secret "prometheus-operator-admission-webhook-tls" not found Jul 07 22:03:11.655757 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:11.655722 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a27db392-9562-49ea-a86c-325f3635da98-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-ckzpq\" (UID: \"a27db392-9562-49ea-a86c-325f3635da98\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" Jul 07 22:03:11.655924 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:11.655849 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 07 22:03:11.655924 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:11.655903 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a27db392-9562-49ea-a86c-325f3635da98-tls-certificates podName:a27db392-9562-49ea-a86c-325f3635da98 nodeName:}" failed. No retries permitted until 2026-07-07 22:03:12.65589003 +0000 UTC m=+62.219988433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a27db392-9562-49ea-a86c-325f3635da98-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-ckzpq" (UID: "a27db392-9562-49ea-a86c-325f3635da98") : secret "prometheus-operator-admission-webhook-tls" not found Jul 07 22:03:11.849733 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:11.849699 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:12.292749 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.292711 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fd9bf9464-zdvmf"] Jul 07 22:03:12.310425 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.310397 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fd9bf9464-zdvmf"] Jul 07 22:03:12.310596 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.310490 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.312792 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.312765 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 22:03:12.312792 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.312766 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 22:03:12.312976 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.312805 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 22:03:12.313633 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.313614 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 22:03:12.313633 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.313624 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8k98k\"" Jul 07 22:03:12.313781 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.313640 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 22:03:12.313781 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.313641 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 22:03:12.313781 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.313711 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 22:03:12.362238 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.362214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-config\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.362355 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.362290 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-service-ca\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.362355 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.362334 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-oauth-config\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.362450 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.362363 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4hp4\" (UniqueName: \"kubernetes.io/projected/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-kube-api-access-h4hp4\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.362505 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.362462 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-serving-cert\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.362505 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.362496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-oauth-serving-cert\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.462844 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.462821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-service-ca\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.462945 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.462860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-oauth-config\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.462945 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.462883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4hp4\" (UniqueName: \"kubernetes.io/projected/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-kube-api-access-h4hp4\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.462945 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.462915 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-serving-cert\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.462945 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.462939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-oauth-serving-cert\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.463126 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.463018 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-config\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.463694 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.463671 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-config\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.463788 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.463668 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-service-ca\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.463788 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.463707 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-oauth-serving-cert\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.465629 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.465609 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-serving-cert\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.465720 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.465702 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-oauth-config\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.471399 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.471377 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4hp4\" (UniqueName: \"kubernetes.io/projected/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-kube-api-access-h4hp4\") pod \"console-5fd9bf9464-zdvmf\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.619571 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.619507 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:12.664565 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.664535 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a27db392-9562-49ea-a86c-325f3635da98-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-ckzpq\" (UID: \"a27db392-9562-49ea-a86c-325f3635da98\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" Jul 07 22:03:12.667587 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.667561 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a27db392-9562-49ea-a86c-325f3635da98-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-ckzpq\" (UID: \"a27db392-9562-49ea-a86c-325f3635da98\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" Jul 07 22:03:12.773581 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.773546 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" Jul 07 22:03:12.784023 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.784002 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fd9bf9464-zdvmf"] Jul 07 22:03:12.787642 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:12.787619 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3edf1e_8f24_4bd1_bb51_ff81e95f4b2b.slice/crio-431fa7d07e6ead32b9974e4e5750d7efa8b9948bed2af32d056a511ecebbcc9e WatchSource:0}: Error finding container 431fa7d07e6ead32b9974e4e5750d7efa8b9948bed2af32d056a511ecebbcc9e: Status 404 returned error can't find the container with id 431fa7d07e6ead32b9974e4e5750d7efa8b9948bed2af32d056a511ecebbcc9e Jul 07 22:03:12.910596 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:12.910500 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq"] Jul 07 22:03:12.913457 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:12.913432 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27db392_9562_49ea_a86c_325f3635da98.slice/crio-a538bb6adedc0c8d936c583ebeb84ead21789bf2a5088a67b7a69a4b79fda7c9 WatchSource:0}: Error finding container a538bb6adedc0c8d936c583ebeb84ead21789bf2a5088a67b7a69a4b79fda7c9: Status 404 returned error can't find the container with id a538bb6adedc0c8d936c583ebeb84ead21789bf2a5088a67b7a69a4b79fda7c9 Jul 07 22:03:13.319807 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:13.319732 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" event={"ID":"a27db392-9562-49ea-a86c-325f3635da98","Type":"ContainerStarted","Data":"a538bb6adedc0c8d936c583ebeb84ead21789bf2a5088a67b7a69a4b79fda7c9"} Jul 07 22:03:13.321164 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:13.321139 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5zv97" event={"ID":"0fdba9d1-084e-419a-badb-52a97c802694","Type":"ContainerStarted","Data":"6128057aa166c3af76a22699c9ae8a3c21b82315a7f27c5bc45c4d79f2001cc6"} Jul 07 22:03:13.322614 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:13.322587 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vjl8c" event={"ID":"96de9fd5-aa10-4eca-ac08-b8e51cac60b1","Type":"ContainerStarted","Data":"56d7ca577b8ed7a9906b0712bbf01cab9a658bf525f8c54598438504a18447eb"} Jul 07 22:03:13.324175 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:13.324152 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-km8w9" event={"ID":"f8b95043-d806-49a7-baeb-451b8d7fac47","Type":"ContainerStarted","Data":"8622ce6a4f0ef5f13a7cbad042c732e75b72781578f7d41fdefbcc088c16402b"} Jul 07 22:03:13.324258 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:13.324199 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-km8w9" event={"ID":"f8b95043-d806-49a7-baeb-451b8d7fac47","Type":"ContainerStarted","Data":"e7c46d881cc3270a710ae20795aa38f8060f00d5f3d869634a757310357e7f10"} Jul 07 22:03:13.324355 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:13.324332 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:13.325496 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:13.325476 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fd9bf9464-zdvmf" event={"ID":"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b","Type":"ContainerStarted","Data":"431fa7d07e6ead32b9974e4e5750d7efa8b9948bed2af32d056a511ecebbcc9e"} Jul 07 22:03:13.336640 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:13.336604 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5zv97" podStartSLOduration=2.244483142 podStartE2EDuration="8.336591934s" totalImagesPullingTime="6.092108792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:13.336138248 +0000 UTC m=+62.900236673" watchObservedRunningTime="2026-07-07 22:03:13.336591934 +0000 UTC m=+62.900690360" Jul 07 22:03:13.353417 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:13.353376 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-km8w9" podStartSLOduration=2.315582881 podStartE2EDuration="8.3533642s" totalImagesPullingTime="6.037781319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:13.352975143 +0000 UTC m=+62.917073570" watchObservedRunningTime="2026-07-07 22:03:13.3533642 +0000 UTC m=+62.917462626" Jul 07 22:03:13.850058 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:13.850017 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:13.902865 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:13.902837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:15.296184 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.296149 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:15.334344 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.334285 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" event={"ID":"a27db392-9562-49ea-a86c-325f3635da98","Type":"ContainerStarted","Data":"2bede6d1abaa0c12d9baf9a0b59e89f68f4baf0a6b451c1480f8224ac85e1fa1"} Jul 07 22:03:15.334656 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.334585 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" Jul 07 22:03:15.339466 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.339429 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vjl8c" event={"ID":"96de9fd5-aa10-4eca-ac08-b8e51cac60b1","Type":"ContainerStarted","Data":"708e7f7373f5b982c1c996a498e5934b7b0ba2bd1557eff7afc53bbff668059c"} Jul 07 22:03:15.340255 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.340235 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" Jul 07 22:03:15.349242 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.349195 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ckzpq" podStartSLOduration=3.575274323 podStartE2EDuration="5.349178622s" totalImagesPullingTime="1.773904299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:15.348576515 +0000 UTC m=+64.912674941" watchObservedRunningTime="2026-07-07 22:03:15.349178622 +0000 UTC m=+64.913277050" Jul 07 22:03:15.354971 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.354949 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:15.589367 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.589324 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:03:15.589559 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.589411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:03:15.591657 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.591627 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 22:03:15.591657 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.591652 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 22:03:15.609541 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.602415 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60d1d68e-fcca-4c49-9d19-91608847c10b-original-pull-secret\") pod \"global-pull-secret-syncer-59xcs\" (UID: \"60d1d68e-fcca-4c49-9d19-91608847c10b\") " pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:03:15.609541 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.606105 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af520e56-7ba8-4259-a907-7a5a28c41e5b-metrics-certs\") pod \"network-metrics-daemon-k9ljx\" (UID: \"af520e56-7ba8-4259-a907-7a5a28c41e5b\") " pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:03:15.673980 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.673945 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59xcs" Jul 07 22:03:15.684846 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.684817 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cx84f\"" Jul 07 22:03:15.690796 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.690771 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzfw8\" (UniqueName: \"kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8\") pod \"network-check-target-qsxfk\" (UID: \"9b236333-c490-4f99-b91f-d44b01461c72\") " pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:03:15.692710 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.692692 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k9ljx" Jul 07 22:03:15.692710 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.692703 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 22:03:15.703492 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.703474 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 22:03:15.715356 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.715331 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzfw8\" (UniqueName: \"kubernetes.io/projected/9b236333-c490-4f99-b91f-d44b01461c72-kube-api-access-lzfw8\") pod \"network-check-target-qsxfk\" (UID: \"9b236333-c490-4f99-b91f-d44b01461c72\") " pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:03:15.967739 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.967660 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:15.993008 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:15.992980 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jvn48\"" Jul 07 22:03:16.000976 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.000953 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:03:16.023032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.023009 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d"] Jul 07 22:03:16.026157 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.026139 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.028685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.028641 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-n6lks\"" Jul 07 22:03:16.028801 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.028769 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 22:03:16.028866 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.028805 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 22:03:16.028937 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.028912 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 22:03:16.030369 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.030309 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 22:03:16.030473 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.030415 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 22:03:16.035962 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.035940 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d"] Jul 07 22:03:16.054220 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.054190 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:16.093383 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.093349 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/67fc7ebc-6c92-41b7-9122-58b183997f93-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.093555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.093392 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67fc7ebc-6c92-41b7-9122-58b183997f93-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.093555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.093460 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42qc8\" (UniqueName: \"kubernetes.io/projected/67fc7ebc-6c92-41b7-9122-58b183997f93-kube-api-access-42qc8\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.093657 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.093550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67fc7ebc-6c92-41b7-9122-58b183997f93-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.194781 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.194747 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/67fc7ebc-6c92-41b7-9122-58b183997f93-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.194781 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.194783 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67fc7ebc-6c92-41b7-9122-58b183997f93-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.195011 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.194803 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42qc8\" (UniqueName: \"kubernetes.io/projected/67fc7ebc-6c92-41b7-9122-58b183997f93-kube-api-access-42qc8\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.195011 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.194830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67fc7ebc-6c92-41b7-9122-58b183997f93-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.195976 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.195946 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67fc7ebc-6c92-41b7-9122-58b183997f93-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.197671 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.197649 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67fc7ebc-6c92-41b7-9122-58b183997f93-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.197790 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.197679 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/67fc7ebc-6c92-41b7-9122-58b183997f93-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.204774 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.204754 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42qc8\" (UniqueName: \"kubernetes.io/projected/67fc7ebc-6c92-41b7-9122-58b183997f93-kube-api-access-42qc8\") pod \"prometheus-operator-567f7b8b7c-tnw2d\" (UID: \"67fc7ebc-6c92-41b7-9122-58b183997f93\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.229012 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.228948 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:16.264750 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.264726 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hk92b" Jul 07 22:03:16.338164 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.338135 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" Jul 07 22:03:16.400774 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.400747 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:16.501874 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.501802 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:16.864432 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.864399 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:16.967278 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:16.967245 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:17.681818 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:17.681735 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k9ljx"] Jul 07 22:03:17.686935 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:17.686885 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf520e56_7ba8_4259_a907_7a5a28c41e5b.slice/crio-566feda25e5c92778204a61f10f083a48b0830665f91cfcfb41aca64c1f3eea5 WatchSource:0}: Error finding container 566feda25e5c92778204a61f10f083a48b0830665f91cfcfb41aca64c1f3eea5: Status 404 returned error can't find the container with id 566feda25e5c92778204a61f10f083a48b0830665f91cfcfb41aca64c1f3eea5 Jul 07 22:03:17.731308 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:17.731279 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qsxfk"] Jul 07 22:03:17.734030 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:17.734006 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b236333_c490_4f99_b91f_d44b01461c72.slice/crio-b66d096b72dc3158dd6027f8582e8fae017ebe062e828faf71b500e9fd40ffde WatchSource:0}: Error finding container b66d096b72dc3158dd6027f8582e8fae017ebe062e828faf71b500e9fd40ffde: Status 404 returned error can't find the container with id b66d096b72dc3158dd6027f8582e8fae017ebe062e828faf71b500e9fd40ffde Jul 07 22:03:17.900929 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:17.900903 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-59xcs"] Jul 07 22:03:17.904114 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:17.904089 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d"] Jul 07 22:03:17.904561 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:17.904532 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60d1d68e_fcca_4c49_9d19_91608847c10b.slice/crio-7db6f02b08b091734df12c47b2b00d137f4080d61ceaf46431f89794f48060c1 WatchSource:0}: Error finding container 7db6f02b08b091734df12c47b2b00d137f4080d61ceaf46431f89794f48060c1: Status 404 returned error can't find the container with id 7db6f02b08b091734df12c47b2b00d137f4080d61ceaf46431f89794f48060c1 Jul 07 22:03:17.907069 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:17.907045 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67fc7ebc_6c92_41b7_9122_58b183997f93.slice/crio-396125f1cae35f551632bfa8501ed629f0cd579e74dce4bec7b5e4a735d773c7 WatchSource:0}: Error finding container 396125f1cae35f551632bfa8501ed629f0cd579e74dce4bec7b5e4a735d773c7: Status 404 returned error can't find the container with id 396125f1cae35f551632bfa8501ed629f0cd579e74dce4bec7b5e4a735d773c7 Jul 07 22:03:18.354261 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:18.354220 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vjl8c" event={"ID":"96de9fd5-aa10-4eca-ac08-b8e51cac60b1","Type":"ContainerStarted","Data":"04ee82d6026adc9a7e168aac41af3b44893365f7f212c943926980744ad88006"} Jul 07 22:03:18.356076 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:18.356036 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fd9bf9464-zdvmf" event={"ID":"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b","Type":"ContainerStarted","Data":"eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab"} Jul 07 22:03:18.357243 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:18.357213 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k9ljx" event={"ID":"af520e56-7ba8-4259-a907-7a5a28c41e5b","Type":"ContainerStarted","Data":"566feda25e5c92778204a61f10f083a48b0830665f91cfcfb41aca64c1f3eea5"} Jul 07 22:03:18.358444 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:18.358414 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" event={"ID":"67fc7ebc-6c92-41b7-9122-58b183997f93","Type":"ContainerStarted","Data":"396125f1cae35f551632bfa8501ed629f0cd579e74dce4bec7b5e4a735d773c7"} Jul 07 22:03:18.359753 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:18.359729 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-59xcs" event={"ID":"60d1d68e-fcca-4c49-9d19-91608847c10b","Type":"ContainerStarted","Data":"7db6f02b08b091734df12c47b2b00d137f4080d61ceaf46431f89794f48060c1"} Jul 07 22:03:18.360999 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:18.360977 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qsxfk" event={"ID":"9b236333-c490-4f99-b91f-d44b01461c72","Type":"ContainerStarted","Data":"b66d096b72dc3158dd6027f8582e8fae017ebe062e828faf71b500e9fd40ffde"} Jul 07 22:03:18.371444 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:18.371399 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vjl8c" podStartSLOduration=2.309738736 podStartE2EDuration="13.37138375s" totalImagesPullingTime="11.061645014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:18.371309759 +0000 UTC m=+67.935408196" watchObservedRunningTime="2026-07-07 22:03:18.37138375 +0000 UTC m=+67.935482183" Jul 07 22:03:18.392580 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:18.392535 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fd9bf9464-zdvmf" podStartSLOduration=1.671994658 podStartE2EDuration="6.392509483s" totalImagesPullingTime="4.720514825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:18.391449151 +0000 UTC m=+67.955547572" watchObservedRunningTime="2026-07-07 22:03:18.392509483 +0000 UTC m=+67.956607908" Jul 07 22:03:18.782884 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:18.782790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:18.882860 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:18.882831 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:19.071947 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.070882 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f8449ff65-tlx5p"] Jul 07 22:03:19.073209 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.073178 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.080824 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.080798 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 22:03:19.082472 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.082415 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f8449ff65-tlx5p"] Jul 07 22:03:19.123131 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.123101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-oauth-config\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.123297 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.123147 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-service-ca\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.123297 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.123217 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-serving-cert\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.123297 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.123233 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-console-config\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.123297 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.123256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg42k\" (UniqueName: \"kubernetes.io/projected/3a83ace4-c925-432b-aefd-81ef71c51284-kube-api-access-rg42k\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.123297 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.123292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-oauth-serving-cert\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.123644 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.123312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-trusted-ca-bundle\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.227723 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.224782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-oauth-config\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.227723 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.224830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-service-ca\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.227723 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.224878 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-serving-cert\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.227723 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.224910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-console-config\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.227723 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.224944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg42k\" (UniqueName: \"kubernetes.io/projected/3a83ace4-c925-432b-aefd-81ef71c51284-kube-api-access-rg42k\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.227723 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.224981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-oauth-serving-cert\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.227723 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.225013 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-trusted-ca-bundle\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.227723 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.225880 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-trusted-ca-bundle\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.227723 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.226852 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-console-config\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.227723 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.227414 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-service-ca\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.229396 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.229366 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-oauth-serving-cert\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.230809 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.230759 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-serving-cert\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.238276 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.238235 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg42k\" (UniqueName: \"kubernetes.io/projected/3a83ace4-c925-432b-aefd-81ef71c51284-kube-api-access-rg42k\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.246982 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.246941 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-oauth-config\") pod \"console-6f8449ff65-tlx5p\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.372819 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.372783 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k9ljx" event={"ID":"af520e56-7ba8-4259-a907-7a5a28c41e5b","Type":"ContainerStarted","Data":"1cfaf0ce858b236aee24e29a3476896ba3d8971d553b001d4a7ab22cd8e8f5f1"} Jul 07 22:03:19.388469 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.388443 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:19.619359 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.619331 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f8449ff65-tlx5p"] Jul 07 22:03:19.622131 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:19.622098 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a83ace4_c925_432b_aefd_81ef71c51284.slice/crio-3dd2c631506e40a8cecfcaced269417fe2b97e0543274575e2249d26b2b2bd14 WatchSource:0}: Error finding container 3dd2c631506e40a8cecfcaced269417fe2b97e0543274575e2249d26b2b2bd14: Status 404 returned error can't find the container with id 3dd2c631506e40a8cecfcaced269417fe2b97e0543274575e2249d26b2b2bd14 Jul 07 22:03:19.856946 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.856875 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:19.957322 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:19.957247 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:20.378960 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:20.378433 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k9ljx" event={"ID":"af520e56-7ba8-4259-a907-7a5a28c41e5b","Type":"ContainerStarted","Data":"ff62a79aa4d983c7719d7dc81fa0b843e7cdba831202803cad7d37d4919cc88a"} Jul 07 22:03:20.380912 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:20.380874 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" event={"ID":"67fc7ebc-6c92-41b7-9122-58b183997f93","Type":"ContainerStarted","Data":"e74fa032cc5c1ff87246986c218a78cb7c1efdbdd5fd5dfea65278bb8d241e80"} Jul 07 22:03:20.381059 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:20.380928 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" event={"ID":"67fc7ebc-6c92-41b7-9122-58b183997f93","Type":"ContainerStarted","Data":"5697469fe55e1285b5c93874a6635a185e1f1859cdeb63bb890f02b85703a219"} Jul 07 22:03:20.382780 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:20.382756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f8449ff65-tlx5p" event={"ID":"3a83ace4-c925-432b-aefd-81ef71c51284","Type":"ContainerStarted","Data":"9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14"} Jul 07 22:03:20.382895 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:20.382794 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f8449ff65-tlx5p" event={"ID":"3a83ace4-c925-432b-aefd-81ef71c51284","Type":"ContainerStarted","Data":"3dd2c631506e40a8cecfcaced269417fe2b97e0543274575e2249d26b2b2bd14"} Jul 07 22:03:20.420923 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:20.420866 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k9ljx" podStartSLOduration=69.240968048 podStartE2EDuration="1m10.420846321s" totalImagesPullingTime="1.179878273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:02:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:20.397261884 +0000 UTC m=+69.961360309" watchObservedRunningTime="2026-07-07 22:03:20.420846321 +0000 UTC m=+69.984944750" Jul 07 22:03:20.437373 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:20.437318 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tnw2d" podStartSLOduration=2.8506607280000003 podStartE2EDuration="4.43730008s" totalImagesPullingTime="1.586639352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:20.436869543 +0000 UTC m=+70.000967969" watchObservedRunningTime="2026-07-07 22:03:20.43730008 +0000 UTC m=+70.001398507" Jul 07 22:03:20.437601 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:20.437439 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f8449ff65-tlx5p" podStartSLOduration=1.437429864 podStartE2EDuration="1.437429864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:20.420676822 +0000 UTC m=+69.984775248" watchObservedRunningTime="2026-07-07 22:03:20.437429864 +0000 UTC m=+70.001528292" Jul 07 22:03:20.501023 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:20.500988 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:20.600036 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:20.599998 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:22.144852 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.144793 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:22.255180 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.255147 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:22.444913 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.444828 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:22.473392 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.473351 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4"] Jul 07 22:03:22.476004 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.475979 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.478347 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.478322 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 22:03:22.478703 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.478632 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 22:03:22.479667 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.479645 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4x7rz\"" Jul 07 22:03:22.490364 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.490340 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4"] Jul 07 22:03:22.502196 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.502174 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg"] Jul 07 22:03:22.504667 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.504645 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.507002 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.506982 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 22:03:22.507104 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.507009 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-26flx\"" Jul 07 22:03:22.507372 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.507352 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 22:03:22.507532 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.507400 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 22:03:22.512245 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.512225 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6f2th"] Jul 07 22:03:22.514726 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.514708 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.517046 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.517027 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 22:03:22.517143 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.517056 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 22:03:22.517143 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.517130 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 22:03:22.517497 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.517482 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wvf4z\"" Jul 07 22:03:22.523401 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.523376 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg"] Jul 07 22:03:22.550382 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.550353 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81ad86f9-d1db-4503-aab2-44589cb920c5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.550382 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.550385 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81ad86f9-d1db-4503-aab2-44589cb920c5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.550610 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.550420 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/81ad86f9-d1db-4503-aab2-44589cb920c5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.550610 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.550451 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.550610 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.550501 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.550610 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.550596 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.550764 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.550632 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.550764 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.550668 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl7ls\" (UniqueName: \"kubernetes.io/projected/81ad86f9-d1db-4503-aab2-44589cb920c5-kube-api-access-tl7ls\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.550764 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.550711 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lfm2\" (UniqueName: \"kubernetes.io/projected/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-api-access-8lfm2\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.550764 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.550753 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.573961 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.573931 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:22.620501 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.620468 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:22.620696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.620549 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:22.626924 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.626893 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:22.650857 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.650821 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:22.651162 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651137 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-tls\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.651254 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651180 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.651254 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651210 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.651254 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/55eb3979-c8a2-4871-8dcb-938f02ee61e5-root\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.651404 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651267 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.651404 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651294 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl7ls\" (UniqueName: \"kubernetes.io/projected/81ad86f9-d1db-4503-aab2-44589cb920c5-kube-api-access-tl7ls\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.651404 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lfm2\" (UniqueName: \"kubernetes.io/projected/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-api-access-8lfm2\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.651404 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.651404 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651392 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55eb3979-c8a2-4871-8dcb-938f02ee61e5-metrics-client-ca\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.651677 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651423 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.651677 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651458 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55eb3979-c8a2-4871-8dcb-938f02ee61e5-sys\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.651677 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651490 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.651677 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81ad86f9-d1db-4503-aab2-44589cb920c5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.651677 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651575 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.651677 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81ad86f9-d1db-4503-aab2-44589cb920c5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.651677 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651614 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-wtmp\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.651677 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651645 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2mvn\" (UniqueName: \"kubernetes.io/projected/55eb3979-c8a2-4871-8dcb-938f02ee61e5-kube-api-access-z2mvn\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.651677 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651678 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-textfile\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.652092 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:22.651697 2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 07 22:03:22.652092 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651711 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/81ad86f9-d1db-4503-aab2-44589cb920c5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.652092 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:22.651753 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-tls podName:00b7c58f-c0ac-4597-a833-d9b7fc4aecd3 nodeName:}" failed. No retries permitted until 2026-07-07 22:03:23.151738653 +0000 UTC m=+72.715837056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-zf9cg" (UID: "00b7c58f-c0ac-4597-a833-d9b7fc4aecd3") : secret "kube-state-metrics-tls" not found Jul 07 22:03:22.652092 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.651774 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.652277 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.652227 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.652424 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.652381 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.653323 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.653261 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81ad86f9-d1db-4503-aab2-44589cb920c5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.654874 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.654853 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.654959 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.654876 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/81ad86f9-d1db-4503-aab2-44589cb920c5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.655001 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.654951 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81ad86f9-d1db-4503-aab2-44589cb920c5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.660508 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.660487 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lfm2\" (UniqueName: \"kubernetes.io/projected/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-api-access-8lfm2\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:22.660645 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.660628 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl7ls\" (UniqueName: \"kubernetes.io/projected/81ad86f9-d1db-4503-aab2-44589cb920c5-kube-api-access-tl7ls\") pod \"openshift-state-metrics-54cf96dfcf-kf2d4\" (UID: \"81ad86f9-d1db-4503-aab2-44589cb920c5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.711685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.711650 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:22.753051 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.753021 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-wtmp\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.753166 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.753099 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2mvn\" (UniqueName: \"kubernetes.io/projected/55eb3979-c8a2-4871-8dcb-938f02ee61e5-kube-api-access-z2mvn\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.753166 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.753148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-textfile\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.753278 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.753200 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-tls\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.753278 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.753209 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-wtmp\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.753278 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.753236 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/55eb3979-c8a2-4871-8dcb-938f02ee61e5-root\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.753413 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.753303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.753413 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.753310 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/55eb3979-c8a2-4871-8dcb-938f02ee61e5-root\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.753413 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.753354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55eb3979-c8a2-4871-8dcb-938f02ee61e5-metrics-client-ca\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.753413 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.753408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55eb3979-c8a2-4871-8dcb-938f02ee61e5-sys\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.753641 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.753442 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.754042 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.754019 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.754137 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.754093 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55eb3979-c8a2-4871-8dcb-938f02ee61e5-sys\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.754354 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.754331 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-textfile\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.754494 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.754475 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55eb3979-c8a2-4871-8dcb-938f02ee61e5-metrics-client-ca\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.756440 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.756411 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-tls\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.757094 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.756687 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55eb3979-c8a2-4871-8dcb-938f02ee61e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.768126 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.768075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2mvn\" (UniqueName: \"kubernetes.io/projected/55eb3979-c8a2-4871-8dcb-938f02ee61e5-kube-api-access-z2mvn\") pod \"node-exporter-6f2th\" (UID: \"55eb3979-c8a2-4871-8dcb-938f02ee61e5\") " pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.788143 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.788122 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" Jul 07 22:03:22.808314 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.808288 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:22.828352 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.827374 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6f2th" Jul 07 22:03:22.843242 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:22.843203 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55eb3979_c8a2_4871_8dcb_938f02ee61e5.slice/crio-a9eb65d3c1eea6d96e3f6b8229b4794e7077008f25f3046df953ec0037f342de WatchSource:0}: Error finding container a9eb65d3c1eea6d96e3f6b8229b4794e7077008f25f3046df953ec0037f342de: Status 404 returned error can't find the container with id a9eb65d3c1eea6d96e3f6b8229b4794e7077008f25f3046df953ec0037f342de Jul 07 22:03:22.910929 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.910896 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:22.927762 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:22.927734 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4"] Jul 07 22:03:22.930777 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:22.930756 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ad86f9_d1db_4503_aab2_44589cb920c5.slice/crio-e7c126c28c8844b5db8090882d02d5383558e25d38c21c5c61c23ee8a82d36f4 WatchSource:0}: Error finding container e7c126c28c8844b5db8090882d02d5383558e25d38c21c5c61c23ee8a82d36f4: Status 404 returned error can't find the container with id e7c126c28c8844b5db8090882d02d5383558e25d38c21c5c61c23ee8a82d36f4 Jul 07 22:03:23.009042 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.009010 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:23.113332 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.113300 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:23.156830 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.156800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:23.159292 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.159275 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/00b7c58f-c0ac-4597-a833-d9b7fc4aecd3-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zf9cg\" (UID: \"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:23.245772 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.245671 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:23.330972 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.330945 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-km8w9" Jul 07 22:03:23.396243 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.396191 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qsxfk" event={"ID":"9b236333-c490-4f99-b91f-d44b01461c72","Type":"ContainerStarted","Data":"5fff9338c66def29457e8236661cad19690f73737232f0167b5857b6566260a3"} Jul 07 22:03:23.396424 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.396379 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:03:23.397725 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.397694 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f2th" event={"ID":"55eb3979-c8a2-4871-8dcb-938f02ee61e5","Type":"ContainerStarted","Data":"a9eb65d3c1eea6d96e3f6b8229b4794e7077008f25f3046df953ec0037f342de"} Jul 07 22:03:23.399503 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.399476 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-59xcs" event={"ID":"60d1d68e-fcca-4c49-9d19-91608847c10b","Type":"ContainerStarted","Data":"05e63b506b1162bb5aa610839ef94f12d2ca088fb72f69a2592b65e92c917e99"} Jul 07 22:03:23.401726 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.401689 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" event={"ID":"81ad86f9-d1db-4503-aab2-44589cb920c5","Type":"ContainerStarted","Data":"fbdc9f88ec37cf6350761a5450e29714031ba9b982d82ee461f1f014dd5a367c"} Jul 07 22:03:23.401824 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.401756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" event={"ID":"81ad86f9-d1db-4503-aab2-44589cb920c5","Type":"ContainerStarted","Data":"6ae27e9583cd93b68f02cd7674260a493fcb24a92141b9298a07d5f1c043736f"} Jul 07 22:03:23.401824 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.401787 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" event={"ID":"81ad86f9-d1db-4503-aab2-44589cb920c5","Type":"ContainerStarted","Data":"e7c126c28c8844b5db8090882d02d5383558e25d38c21c5c61c23ee8a82d36f4"} Jul 07 22:03:23.407225 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.406577 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:03:23.412066 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.412011 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qsxfk" podStartSLOduration=67.440238796 podStartE2EDuration="1m12.411992431s" totalImagesPullingTime="4.971753635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:23.410886503 +0000 UTC m=+72.974984928" watchObservedRunningTime="2026-07-07 22:03:23.411992431 +0000 UTC m=+72.976090859" Jul 07 22:03:23.414474 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.414454 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" Jul 07 22:03:23.442568 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.442492 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-59xcs" podStartSLOduration=67.637431492 podStartE2EDuration="1m12.442472598s" totalImagesPullingTime="4.805041106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:23.441889926 +0000 UTC m=+73.005988354" watchObservedRunningTime="2026-07-07 22:03:23.442472598 +0000 UTC m=+73.006571024" Jul 07 22:03:23.474202 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.474174 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:23.526745 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.526711 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 22:03:23.530631 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.530605 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.533247 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.533138 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 22:03:23.534716 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.533709 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 22:03:23.534716 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.533839 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 22:03:23.534716 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.533892 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 22:03:23.534716 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.534100 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 22:03:23.534716 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.534137 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 22:03:23.534716 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.534103 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 22:03:23.534716 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.534220 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6j8h4\"" Jul 07 22:03:23.534716 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.534100 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 22:03:23.534716 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.534493 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 22:03:23.550995 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.550970 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 22:03:23.557103 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.557075 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:23.578075 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.578050 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg"] Jul 07 22:03:23.580456 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:23.580432 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b7c58f_c0ac_4597_a833_d9b7fc4aecd3.slice/crio-b553d8eabffb1385f02a5bdf228dd7962a109a983cdbf024f26cdf87ed7a16ce WatchSource:0}: Error finding container b553d8eabffb1385f02a5bdf228dd7962a109a983cdbf024f26cdf87ed7a16ce: Status 404 returned error can't find the container with id b553d8eabffb1385f02a5bdf228dd7962a109a983cdbf024f26cdf87ed7a16ce Jul 07 22:03:23.662090 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662055 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-volume\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662252 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662252 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662125 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662252 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662151 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662252 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662188 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4jpl\" (UniqueName: \"kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-kube-api-access-n4jpl\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662461 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662264 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662461 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662324 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662461 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662381 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662641 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662453 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-out\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662641 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662641 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662503 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662641 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.662641 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.662617 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-web-config\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.763943 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.763854 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-out\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.763943 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.763899 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.764166 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.764087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.764166 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.764147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.764275 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.764178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-web-config\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.764275 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.764209 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-volume\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.764275 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.764234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.764417 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.764258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.764417 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.764314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.764417 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.764345 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4jpl\" (UniqueName: \"kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-kube-api-access-n4jpl\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.764417 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.764400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.764645 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.764451 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.764645 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.764508 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.765303 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:23.765278 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-trusted-ca-bundle podName:e002aec2-e4a5-4592-9f6c-692c0b7624a2 nodeName:}" failed. No retries permitted until 2026-07-07 22:03:24.2652536 +0000 UTC m=+73.829352021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2") : configmap references non-existent config key: ca-bundle.crt Jul 07 22:03:23.765842 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.765818 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.766319 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:23.766297 2580 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 07 22:03:23.766417 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:23.766371 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-main-tls podName:e002aec2-e4a5-4592-9f6c-692c0b7624a2 nodeName:}" failed. No retries permitted until 2026-07-07 22:03:24.266354937 +0000 UTC m=+73.830453376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2") : secret "alertmanager-main-tls" not found Jul 07 22:03:23.766765 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.766740 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.767348 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.767301 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-web-config\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.768498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.768446 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.769769 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.769707 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-volume\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.770535 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.770497 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.770891 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.770829 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.771939 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.771913 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.772288 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.772267 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-out\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.776637 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.776605 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4jpl\" (UniqueName: \"kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-kube-api-access-n4jpl\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:23.777073 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:23.777032 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:24.158640 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.158607 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:24.267294 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.267260 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:24.267454 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.267360 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:24.268116 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.268096 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:24.269729 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.269703 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:24.408214 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.408175 2580 generic.go:356] "Generic (PLEG): container finished" podID="55eb3979-c8a2-4871-8dcb-938f02ee61e5" containerID="9774e29756d2a52d9ecc4e2674338737f5802394e6da0599877316cc63132530" exitCode=0 Jul 07 22:03:24.408388 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.408266 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f2th" event={"ID":"55eb3979-c8a2-4871-8dcb-938f02ee61e5","Type":"ContainerDied","Data":"9774e29756d2a52d9ecc4e2674338737f5802394e6da0599877316cc63132530"} Jul 07 22:03:24.410809 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.410772 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" event={"ID":"81ad86f9-d1db-4503-aab2-44589cb920c5","Type":"ContainerStarted","Data":"4d5e6451c4eecfa0d244f2e3cfc5c95c6869be2db4a9a830e39d182e6a24cdf1"} Jul 07 22:03:24.412181 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.412153 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" event={"ID":"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3","Type":"ContainerStarted","Data":"b553d8eabffb1385f02a5bdf228dd7962a109a983cdbf024f26cdf87ed7a16ce"} Jul 07 22:03:24.425623 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.425596 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-74fc86b697-n572z"] Jul 07 22:03:24.432178 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.431175 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.433879 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.433850 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 22:03:24.433987 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.433918 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 22:03:24.433987 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.433852 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 22:03:24.434219 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.434203 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 22:03:24.434292 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.434277 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-77dgl\"" Jul 07 22:03:24.434356 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.434345 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 22:03:24.434394 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.434344 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3ocqnqhssvumm\"" Jul 07 22:03:24.441002 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.440978 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-74fc86b697-n572z"] Jul 07 22:03:24.448552 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.448304 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:03:24.468793 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.468767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.469078 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.469056 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d746d876-e77b-493c-bcf8-3fe98d15d59f-metrics-client-ca\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.469358 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.469320 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.469603 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.469578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-tls\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.469677 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.469621 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.469926 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.469908 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-grpc-tls\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.471039 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.471015 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.471196 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.471179 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv7nm\" (UniqueName: \"kubernetes.io/projected/d746d876-e77b-493c-bcf8-3fe98d15d59f-kube-api-access-jv7nm\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.481825 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.481781 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kf2d4" podStartSLOduration=1.346351752 podStartE2EDuration="2.481767106s" totalImagesPullingTime="1.135415354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:24.481749758 +0000 UTC m=+74.045848185" watchObservedRunningTime="2026-07-07 22:03:24.481767106 +0000 UTC m=+74.045865531" Jul 07 22:03:24.572131 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.572047 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-tls\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.572131 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.572082 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.572335 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.572132 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-grpc-tls\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.572335 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.572185 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.572335 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.572214 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv7nm\" (UniqueName: \"kubernetes.io/projected/d746d876-e77b-493c-bcf8-3fe98d15d59f-kube-api-access-jv7nm\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.572335 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.572268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.572335 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.572314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d746d876-e77b-493c-bcf8-3fe98d15d59f-metrics-client-ca\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.572706 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.572358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.574082 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.574033 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d746d876-e77b-493c-bcf8-3fe98d15d59f-metrics-client-ca\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.575622 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.575575 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-tls\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.576495 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.576469 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.576929 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.576903 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.577345 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.577318 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.577599 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.577578 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.577868 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.577847 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d746d876-e77b-493c-bcf8-3fe98d15d59f-secret-grpc-tls\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.581180 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.581157 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv7nm\" (UniqueName: \"kubernetes.io/projected/d746d876-e77b-493c-bcf8-3fe98d15d59f-kube-api-access-jv7nm\") pod \"thanos-querier-74fc86b697-n572z\" (UID: \"d746d876-e77b-493c-bcf8-3fe98d15d59f\") " pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.609384 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.609305 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 22:03:24.613098 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:24.613070 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode002aec2_e4a5_4592_9f6c_692c0b7624a2.slice/crio-d1f542a46ce27cac3a10da4c35b4ef7a952c9445439281c14431b6133aed6119 WatchSource:0}: Error finding container d1f542a46ce27cac3a10da4c35b4ef7a952c9445439281c14431b6133aed6119: Status 404 returned error can't find the container with id d1f542a46ce27cac3a10da4c35b4ef7a952c9445439281c14431b6133aed6119 Jul 07 22:03:24.666160 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.666077 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:24.774114 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.774079 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:24.929265 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:24.929239 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-74fc86b697-n572z"] Jul 07 22:03:25.163255 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:25.163224 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd746d876_e77b_493c_bcf8_3fe98d15d59f.slice/crio-f05f91fdd608f1560286a1812de22134dd7773449a8098d02d8fcd1141e28b41 WatchSource:0}: Error finding container f05f91fdd608f1560286a1812de22134dd7773449a8098d02d8fcd1141e28b41: Status 404 returned error can't find the container with id f05f91fdd608f1560286a1812de22134dd7773449a8098d02d8fcd1141e28b41 Jul 07 22:03:25.284836 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.284750 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:25.387662 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.387565 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:25.423714 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.423685 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f2th" event={"ID":"55eb3979-c8a2-4871-8dcb-938f02ee61e5","Type":"ContainerStarted","Data":"2a4460f618ec969e50712d23acefff41518766a36cf290b502de9e9a320b1312"} Jul 07 22:03:25.423852 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.423735 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f2th" event={"ID":"55eb3979-c8a2-4871-8dcb-938f02ee61e5","Type":"ContainerStarted","Data":"39ca0d6646e0f3955efd9639feca88ba5357fd5786de2077d8a36391a56fbccd"} Jul 07 22:03:25.427148 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.427120 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" event={"ID":"d746d876-e77b-493c-bcf8-3fe98d15d59f","Type":"ContainerStarted","Data":"f05f91fdd608f1560286a1812de22134dd7773449a8098d02d8fcd1141e28b41"} Jul 07 22:03:25.429773 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.429749 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerStarted","Data":"d1f542a46ce27cac3a10da4c35b4ef7a952c9445439281c14431b6133aed6119"} Jul 07 22:03:25.433230 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.433200 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" event={"ID":"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3","Type":"ContainerStarted","Data":"aada4e4176fa2d4115d6b9324ed88b256f7720592cfdee06b1eb944980b0183f"} Jul 07 22:03:25.433346 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.433256 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" event={"ID":"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3","Type":"ContainerStarted","Data":"e4d65fc143598c9bb034683d151b7aeb0073898b89a0d812d33b39fddb279e72"} Jul 07 22:03:25.433346 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.433288 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" event={"ID":"00b7c58f-c0ac-4597-a833-d9b7fc4aecd3","Type":"ContainerStarted","Data":"13a578da76f4f76744939a691939e11dc0ade95c000c9ff4a4b7d76a97ca7859"} Jul 07 22:03:25.444676 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.443913 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6f2th" podStartSLOduration=2.277259618 podStartE2EDuration="3.443893368s" totalImagesPullingTime="1.08630929s" totalInitContainerRuntime="80.32446ms" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:25.443116624 +0000 UTC m=+75.007215052" watchObservedRunningTime="2026-07-07 22:03:25.443893368 +0000 UTC m=+75.007991794" Jul 07 22:03:25.462257 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.462199 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zf9cg" podStartSLOduration=1.853407864 podStartE2EDuration="3.462178922s" totalImagesPullingTime="1.608771058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:25.460890602 +0000 UTC m=+75.024989030" watchObservedRunningTime="2026-07-07 22:03:25.462178922 +0000 UTC m=+75.026277350" Jul 07 22:03:25.465637 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.465603 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:25.657408 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:25.657321 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:26.081015 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.080981 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:26.439222 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.439147 2580 generic.go:356] "Generic (PLEG): container finished" podID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerID="04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616" exitCode=0 Jul 07 22:03:26.439596 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.439240 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerDied","Data":"04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616"} Jul 07 22:03:26.862101 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.862062 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-686d5bf558-vkqch"] Jul 07 22:03:26.867773 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.867744 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:26.870018 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.869991 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 22:03:26.870148 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.870106 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qldn5\"" Jul 07 22:03:26.870379 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.870359 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 22:03:26.870458 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.870383 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 22:03:26.870647 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.870628 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-38vuh8gqj6dmd\"" Jul 07 22:03:26.870752 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.870668 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 22:03:26.874948 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.874924 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-686d5bf558-vkqch"] Jul 07 22:03:26.995285 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.995242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/62b999e7-9b38-42be-8c54-4cd8862944bc-metrics-server-audit-profiles\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:26.995474 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.995307 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/62b999e7-9b38-42be-8c54-4cd8862944bc-secret-metrics-server-client-certs\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:26.995474 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.995341 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8rxs\" (UniqueName: \"kubernetes.io/projected/62b999e7-9b38-42be-8c54-4cd8862944bc-kube-api-access-n8rxs\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:26.995474 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.995377 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b999e7-9b38-42be-8c54-4cd8862944bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:26.995474 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.995393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/62b999e7-9b38-42be-8c54-4cd8862944bc-audit-log\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:26.995804 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.995474 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/62b999e7-9b38-42be-8c54-4cd8862944bc-secret-metrics-server-tls\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:26.995804 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:26.995546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62b999e7-9b38-42be-8c54-4cd8862944bc-client-ca-bundle\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.096578 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.096555 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/62b999e7-9b38-42be-8c54-4cd8862944bc-secret-metrics-server-client-certs\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.096661 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.096586 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8rxs\" (UniqueName: \"kubernetes.io/projected/62b999e7-9b38-42be-8c54-4cd8862944bc-kube-api-access-n8rxs\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.096661 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.096617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b999e7-9b38-42be-8c54-4cd8862944bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.096661 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.096635 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/62b999e7-9b38-42be-8c54-4cd8862944bc-audit-log\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.096925 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.096830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/62b999e7-9b38-42be-8c54-4cd8862944bc-secret-metrics-server-tls\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.097000 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.096966 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/62b999e7-9b38-42be-8c54-4cd8862944bc-audit-log\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.097000 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.096990 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62b999e7-9b38-42be-8c54-4cd8862944bc-client-ca-bundle\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.097100 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.097061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/62b999e7-9b38-42be-8c54-4cd8862944bc-metrics-server-audit-profiles\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.097333 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.097306 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b999e7-9b38-42be-8c54-4cd8862944bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.097893 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.097874 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/62b999e7-9b38-42be-8c54-4cd8862944bc-metrics-server-audit-profiles\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.099753 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.099724 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62b999e7-9b38-42be-8c54-4cd8862944bc-client-ca-bundle\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.099843 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.099802 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/62b999e7-9b38-42be-8c54-4cd8862944bc-secret-metrics-server-client-certs\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.099898 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.099869 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/62b999e7-9b38-42be-8c54-4cd8862944bc-secret-metrics-server-tls\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.106782 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.106765 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8rxs\" (UniqueName: \"kubernetes.io/projected/62b999e7-9b38-42be-8c54-4cd8862944bc-kube-api-access-n8rxs\") pod \"metrics-server-686d5bf558-vkqch\" (UID: \"62b999e7-9b38-42be-8c54-4cd8862944bc\") " pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.180776 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.180747 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:27.186685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.186657 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p"] Jul 07 22:03:27.189672 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.189645 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" Jul 07 22:03:27.193545 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.193115 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-zsmp9\"" Jul 07 22:03:27.193545 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.193377 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 22:03:27.200426 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.200336 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p"] Jul 07 22:03:27.201034 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.200607 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:27.300036 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.300008 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-5629p\" (UID: \"2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" Jul 07 22:03:27.300821 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.300794 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:27.301106 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.301079 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtqw8\" (UniqueName: \"kubernetes.io/projected/2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe-kube-api-access-gtqw8\") pod \"monitoring-plugin-d5dcff7cf-5629p\" (UID: \"2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" Jul 07 22:03:27.336248 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.336093 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-686d5bf558-vkqch"] Jul 07 22:03:27.339549 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:27.339493 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b999e7_9b38_42be_8c54_4cd8862944bc.slice/crio-c61311944d189450a3e0a192b86c87090edb62b6c6b0227d1004eaa7edd89ba8 WatchSource:0}: Error finding container c61311944d189450a3e0a192b86c87090edb62b6c6b0227d1004eaa7edd89ba8: Status 404 returned error can't find the container with id c61311944d189450a3e0a192b86c87090edb62b6c6b0227d1004eaa7edd89ba8 Jul 07 22:03:27.402446 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.402372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-5629p\" (UID: \"2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" Jul 07 22:03:27.402446 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.402409 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtqw8\" (UniqueName: \"kubernetes.io/projected/2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe-kube-api-access-gtqw8\") pod \"monitoring-plugin-d5dcff7cf-5629p\" (UID: \"2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" Jul 07 22:03:27.402691 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:27.402540 2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 07 22:03:27.402691 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:27.402615 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe-monitoring-plugin-cert podName:2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe nodeName:}" failed. No retries permitted until 2026-07-07 22:03:27.902594004 +0000 UTC m=+77.466692409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-5629p" (UID: "2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe") : secret "monitoring-plugin-cert" not found Jul 07 22:03:27.410872 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.410851 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtqw8\" (UniqueName: \"kubernetes.io/projected/2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe-kube-api-access-gtqw8\") pod \"monitoring-plugin-d5dcff7cf-5629p\" (UID: \"2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" Jul 07 22:03:27.444182 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.444157 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" event={"ID":"62b999e7-9b38-42be-8c54-4cd8862944bc","Type":"ContainerStarted","Data":"c61311944d189450a3e0a192b86c87090edb62b6c6b0227d1004eaa7edd89ba8"} Jul 07 22:03:27.446156 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.446135 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" event={"ID":"d746d876-e77b-493c-bcf8-3fe98d15d59f","Type":"ContainerStarted","Data":"029f0db963323e5212a0f2bdaa8f08a11540238ad47a4d3e55fa6552b2084c26"} Jul 07 22:03:27.446221 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.446190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" event={"ID":"d746d876-e77b-493c-bcf8-3fe98d15d59f","Type":"ContainerStarted","Data":"451d321a273d0c8621fe14ea3db80c89d9774e6047cfe217c46f2bfa9fdbc779"} Jul 07 22:03:27.446254 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.446239 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" event={"ID":"d746d876-e77b-493c-bcf8-3fe98d15d59f","Type":"ContainerStarted","Data":"c6eecdc8d2a1c3d54701ec8a51f4b1c3289d0182d79f7ec14f2aad4607a6f4d0"} Jul 07 22:03:27.485800 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.485772 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:27.587110 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.587066 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:27.627830 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.627793 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7fdf768446-ttr6s"] Jul 07 22:03:27.631636 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.631616 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.633721 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.633692 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 22:03:27.633844 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.633821 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 22:03:27.633844 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.633820 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 22:03:27.633954 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.633844 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 22:03:27.634008 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.633973 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-g5c6b\"" Jul 07 22:03:27.634204 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.634190 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 22:03:27.641658 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.641630 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 22:03:27.647322 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.647299 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7fdf768446-ttr6s"] Jul 07 22:03:27.679867 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.679798 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:27.704804 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.704776 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97ggv\" (UniqueName: \"kubernetes.io/projected/ffd89220-8f5a-4719-8853-7399c5d299e4-kube-api-access-97ggv\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.704909 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.704813 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd89220-8f5a-4719-8853-7399c5d299e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.704909 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.704837 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.704994 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.704905 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffd89220-8f5a-4719-8853-7399c5d299e4-metrics-client-ca\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.704994 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.704939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-secret-telemeter-client\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.704994 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.704990 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-federate-client-tls\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.705087 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.705051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-telemeter-client-tls\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.705121 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.705083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd89220-8f5a-4719-8853-7399c5d299e4-serving-certs-ca-bundle\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.806060 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.806025 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-telemeter-client-tls\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.806240 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.806073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd89220-8f5a-4719-8853-7399c5d299e4-serving-certs-ca-bundle\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.806240 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.806121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97ggv\" (UniqueName: \"kubernetes.io/projected/ffd89220-8f5a-4719-8853-7399c5d299e4-kube-api-access-97ggv\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.806240 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.806159 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd89220-8f5a-4719-8853-7399c5d299e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.806240 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.806194 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.806240 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.806234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffd89220-8f5a-4719-8853-7399c5d299e4-metrics-client-ca\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.806503 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.806263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-secret-telemeter-client\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.806503 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.806331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-federate-client-tls\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.807035 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.806987 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd89220-8f5a-4719-8853-7399c5d299e4-serving-certs-ca-bundle\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.807186 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.807163 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd89220-8f5a-4719-8853-7399c5d299e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.807440 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.807392 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffd89220-8f5a-4719-8853-7399c5d299e4-metrics-client-ca\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.808987 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.808963 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.809722 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.809677 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-secret-telemeter-client\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.809809 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.809791 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-telemeter-client-tls\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.809875 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.809856 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ffd89220-8f5a-4719-8853-7399c5d299e4-federate-client-tls\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.815390 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.815367 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97ggv\" (UniqueName: \"kubernetes.io/projected/ffd89220-8f5a-4719-8853-7399c5d299e4-kube-api-access-97ggv\") pod \"telemeter-client-7fdf768446-ttr6s\" (UID: \"ffd89220-8f5a-4719-8853-7399c5d299e4\") " pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:27.820303 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.820280 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:27.907047 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.906939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-5629p\" (UID: \"2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" Jul 07 22:03:27.909608 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.909583 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-5629p\" (UID: \"2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" Jul 07 22:03:27.948525 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:27.948445 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" Jul 07 22:03:28.114291 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.113894 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7fdf768446-ttr6s"] Jul 07 22:03:28.126535 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.126481 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" Jul 07 22:03:28.155469 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:28.155434 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd89220_8f5a_4719_8853_7399c5d299e4.slice/crio-5476dab68f86e2dbd99503207ff6ac6bfe699d518db90d701f64ce45a696f833 WatchSource:0}: Error finding container 5476dab68f86e2dbd99503207ff6ac6bfe699d518db90d701f64ce45a696f833: Status 404 returned error can't find the container with id 5476dab68f86e2dbd99503207ff6ac6bfe699d518db90d701f64ce45a696f833 Jul 07 22:03:28.294293 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.293629 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f8449ff65-tlx5p"] Jul 07 22:03:28.312743 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.312663 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p"] Jul 07 22:03:28.327634 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.327602 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-695df9c75f-bjt2l"] Jul 07 22:03:28.330982 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.330958 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.345865 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.345750 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695df9c75f-bjt2l"] Jul 07 22:03:28.413909 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.413874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-serving-cert\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.414078 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.413929 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-config\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.414078 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.413996 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-trusted-ca-bundle\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.414078 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.414050 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-service-ca\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.414239 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.414111 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-oauth-serving-cert\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.414239 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.414176 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwd54\" (UniqueName: \"kubernetes.io/projected/cc49ffba-aa22-4dfc-a061-dd24f3690c30-kube-api-access-hwd54\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.414239 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.414214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-oauth-config\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.453833 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.453750 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" event={"ID":"ffd89220-8f5a-4719-8853-7399c5d299e4","Type":"ContainerStarted","Data":"5476dab68f86e2dbd99503207ff6ac6bfe699d518db90d701f64ce45a696f833"} Jul 07 22:03:28.516568 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.515067 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-oauth-serving-cert\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.516568 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.515134 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwd54\" (UniqueName: \"kubernetes.io/projected/cc49ffba-aa22-4dfc-a061-dd24f3690c30-kube-api-access-hwd54\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.516568 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.515169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-oauth-config\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.516568 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.515217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-serving-cert\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.516568 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.515242 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-config\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.516568 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.515265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-trusted-ca-bundle\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.516568 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.515298 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-service-ca\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.516568 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.516054 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-service-ca\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.517061 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.516625 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-oauth-serving-cert\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.517336 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.517222 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-config\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.517666 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.517642 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-trusted-ca-bundle\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.519443 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.519406 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-oauth-config\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.520128 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.520091 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-serving-cert\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.527535 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.527473 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwd54\" (UniqueName: \"kubernetes.io/projected/cc49ffba-aa22-4dfc-a061-dd24f3690c30-kube-api-access-hwd54\") pod \"console-695df9c75f-bjt2l\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.549718 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.549686 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:28.645842 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.645808 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:28.653243 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.653211 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:28.666912 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:28.666872 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1644a7_f5c5_4fc2_a33b_86bcb1f19cfe.slice/crio-f80c99e55f129eab1daf85e6ca8b866c3245a6dd805634a3d184c935f4c52321 WatchSource:0}: Error finding container f80c99e55f129eab1daf85e6ca8b866c3245a6dd805634a3d184c935f4c52321: Status 404 returned error can't find the container with id f80c99e55f129eab1daf85e6ca8b866c3245a6dd805634a3d184c935f4c52321 Jul 07 22:03:28.706366 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.706321 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:03:28.709751 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.709728 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.713941 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.713913 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 22:03:28.714054 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.713993 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k89n2\"" Jul 07 22:03:28.714221 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.714198 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 22:03:28.714298 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.714238 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 22:03:28.714409 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.714389 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 22:03:28.714456 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.714440 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 22:03:28.714680 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.714661 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 22:03:28.714772 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.714718 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 22:03:28.714856 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.714670 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 22:03:28.714856 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.714850 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 22:03:28.715095 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.715075 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 22:03:28.715427 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.715409 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f6vh8dpimejbh\"" Jul 07 22:03:28.718387 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.718364 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 22:03:28.718851 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.718834 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 22:03:28.720219 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.720181 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 22:03:28.720484 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.720461 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 22:03:28.730819 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.730796 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:03:28.732717 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.732638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:28.822262 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822230 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822429 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822277 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822429 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822429 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822335 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k66ks\" (UniqueName: \"kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-kube-api-access-k66ks\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822429 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822398 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822429 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822415 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822744 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822440 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822744 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822469 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822744 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822485 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822744 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822569 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822744 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822615 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822744 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822649 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822744 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822679 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822744 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822729 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822772 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822798 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-web-config\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822845 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config-out\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.822983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.822892 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.844638 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.844548 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695df9c75f-bjt2l"] Jul 07 22:03:28.924028 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.923988 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924222 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924034 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924222 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924222 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924222 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924222 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924222 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924222 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924204 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924232 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-web-config\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924315 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config-out\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924471 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k66ks\" (UniqueName: \"kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-kube-api-access-k66ks\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924574 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.924600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924602 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.925077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.924625 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.925500 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.925450 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.927197 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.927168 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.927297 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.927255 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.929559 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.929506 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.929816 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.929794 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.930558 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.930206 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.931408 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.930814 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-web-config\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.931408 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.930881 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.931408 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.931114 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.931408 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.931335 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.931748 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.931599 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.932600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.932191 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.932600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.932338 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.934542 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.932791 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.934542 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.933986 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.936028 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.935739 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.936028 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.935913 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.939451 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.939405 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config-out\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.940848 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.940546 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:28.942890 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:28.942870 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k66ks\" (UniqueName: \"kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-kube-api-access-k66ks\") pod \"prometheus-k8s-0\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:29.030620 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.030535 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:29.209119 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:29.209076 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc49ffba_aa22_4dfc_a061_dd24f3690c30.slice/crio-1e804f3469e967db576c62e5cd908fb17857acf64dcc981f9c85d17724488073 WatchSource:0}: Error finding container 1e804f3469e967db576c62e5cd908fb17857acf64dcc981f9c85d17724488073: Status 404 returned error can't find the container with id 1e804f3469e967db576c62e5cd908fb17857acf64dcc981f9c85d17724488073 Jul 07 22:03:29.389007 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.388759 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:29.392167 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.391807 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:03:29.394801 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:03:29.394773 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf5e29f1_0e94_4d25_808e_63b47a5bc85d.slice/crio-a0cc7bd272320c40652e7254241a056184e2b945c89d6cb1651be760b255ec17 WatchSource:0}: Error finding container a0cc7bd272320c40652e7254241a056184e2b945c89d6cb1651be760b255ec17: Status 404 returned error can't find the container with id a0cc7bd272320c40652e7254241a056184e2b945c89d6cb1651be760b255ec17 Jul 07 22:03:29.469866 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.469449 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" event={"ID":"2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe","Type":"ContainerStarted","Data":"f80c99e55f129eab1daf85e6ca8b866c3245a6dd805634a3d184c935f4c52321"} Jul 07 22:03:29.475704 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.475662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695df9c75f-bjt2l" event={"ID":"cc49ffba-aa22-4dfc-a061-dd24f3690c30","Type":"ContainerStarted","Data":"6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e"} Jul 07 22:03:29.475817 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.475721 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695df9c75f-bjt2l" event={"ID":"cc49ffba-aa22-4dfc-a061-dd24f3690c30","Type":"ContainerStarted","Data":"1e804f3469e967db576c62e5cd908fb17857acf64dcc981f9c85d17724488073"} Jul 07 22:03:29.481845 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.481723 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerStarted","Data":"3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19"} Jul 07 22:03:29.481845 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.481796 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerStarted","Data":"332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b"} Jul 07 22:03:29.488909 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.488865 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerStarted","Data":"a0cc7bd272320c40652e7254241a056184e2b945c89d6cb1651be760b255ec17"} Jul 07 22:03:29.493816 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.493741 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" event={"ID":"62b999e7-9b38-42be-8c54-4cd8862944bc","Type":"ContainerStarted","Data":"a1abbdff99d95d585751d4a04d700aa02f090d9da982c11f2ddd180f12ea5e3d"} Jul 07 22:03:29.498535 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.497030 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-695df9c75f-bjt2l" podStartSLOduration=1.497013045 podStartE2EDuration="1.497013045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:29.496010441 +0000 UTC m=+79.060108870" watchObservedRunningTime="2026-07-07 22:03:29.497013045 +0000 UTC m=+79.061111451" Jul 07 22:03:29.506071 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.505655 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" event={"ID":"d746d876-e77b-493c-bcf8-3fe98d15d59f","Type":"ContainerStarted","Data":"f911de6dc1cd8bc9b1776de49c8569693d2b66998eb0df6b8a4cd18171833f1f"} Jul 07 22:03:29.506348 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.506276 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" event={"ID":"d746d876-e77b-493c-bcf8-3fe98d15d59f","Type":"ContainerStarted","Data":"8030e67fab92a2524e3545d7dca1d9631ec57c55a198c862e51029343d42de09"} Jul 07 22:03:29.506617 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.506558 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" event={"ID":"d746d876-e77b-493c-bcf8-3fe98d15d59f","Type":"ContainerStarted","Data":"01bfbd5e95430563168163affaca37c9b48082529bdd3d0a4f6f8f8088b10357"} Jul 07 22:03:29.547359 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:29.547306 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" podStartSLOduration=1.640124595 podStartE2EDuration="3.547282273s" totalImagesPullingTime="1.907157678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:29.518450204 +0000 UTC m=+79.082548630" watchObservedRunningTime="2026-07-07 22:03:29.547282273 +0000 UTC m=+79.111380700" Jul 07 22:03:30.160590 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.160554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:30.224247 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.224210 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:30.511554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.511445 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" event={"ID":"2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe","Type":"ContainerStarted","Data":"d58ebf910ffcefcbe5ea289fbb15bda39801b5f9c468363ce8fc351a5436c0b9"} Jul 07 22:03:30.511985 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.511575 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" Jul 07 22:03:30.515843 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.515807 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerStarted","Data":"9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd"} Jul 07 22:03:30.515972 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.515881 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerStarted","Data":"b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d"} Jul 07 22:03:30.515972 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.515921 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerStarted","Data":"c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77"} Jul 07 22:03:30.515972 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.515966 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerStarted","Data":"41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350"} Jul 07 22:03:30.517684 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.517658 2580 generic.go:356] "Generic (PLEG): container finished" podID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerID="72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5" exitCode=0 Jul 07 22:03:30.517816 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.517667 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" Jul 07 22:03:30.517816 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.517786 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerDied","Data":"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5"} Jul 07 22:03:30.518288 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.518267 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:30.530252 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.530205 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-5629p" podStartSLOduration=1.917573854 podStartE2EDuration="3.530189946s" totalImagesPullingTime="1.612616092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:30.528891794 +0000 UTC m=+80.092990220" watchObservedRunningTime="2026-07-07 22:03:30.530189946 +0000 UTC m=+80.094288371" Jul 07 22:03:30.531378 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.531336 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" podStartSLOduration=3.250609038 podStartE2EDuration="6.531324543s" totalImagesPullingTime="3.280715505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:29.546648847 +0000 UTC m=+79.110747273" watchObservedRunningTime="2026-07-07 22:03:30.531324543 +0000 UTC m=+80.095422964" Jul 07 22:03:30.575406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:30.575347 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.74101514 podStartE2EDuration="7.575325506s" totalImagesPullingTime="3.815141842s" totalInitContainerRuntime="19.168524ms" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:30.573617892 +0000 UTC m=+80.137716321" watchObservedRunningTime="2026-07-07 22:03:30.575325506 +0000 UTC m=+80.139423936" Jul 07 22:03:30.983484 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:30.983460 2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd89220_8f5a_4719_8853_7399c5d299e4.slice/crio-d08188fa7d9647fbd780b47f6a84de8a5bfe6f42daed2995006840794bba1ba0.scope\": RecentStats: unable to find data in memory cache]" Jul 07 22:03:31.526421 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:31.526380 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" event={"ID":"ffd89220-8f5a-4719-8853-7399c5d299e4","Type":"ContainerStarted","Data":"43c6c7fcb553649566e20fb664181ad127e2dbd129bf0a0fa5c0fe4148b2ab74"} Jul 07 22:03:31.526896 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:31.526485 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" event={"ID":"ffd89220-8f5a-4719-8853-7399c5d299e4","Type":"ContainerStarted","Data":"e8dc029cfd22cefd3a80a586a2c2820d91def8f6ea72cee0e8a41fe421c98952"} Jul 07 22:03:31.526896 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:31.526535 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" event={"ID":"ffd89220-8f5a-4719-8853-7399c5d299e4","Type":"ContainerStarted","Data":"d08188fa7d9647fbd780b47f6a84de8a5bfe6f42daed2995006840794bba1ba0"} Jul 07 22:03:31.547800 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:31.547735 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7fdf768446-ttr6s" podStartSLOduration=1.765364458 podStartE2EDuration="4.54771541s" totalImagesPullingTime="2.782350952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:31.54629284 +0000 UTC m=+81.110391289" watchObservedRunningTime="2026-07-07 22:03:31.54771541 +0000 UTC m=+81.111813836" Jul 07 22:03:32.365403 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:32.365367 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695df9c75f-bjt2l"] Jul 07 22:03:34.150547 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:34.150444 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:34.249016 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:34.248981 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:34.544595 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:34.544505 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerStarted","Data":"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1"} Jul 07 22:03:34.544727 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:34.544606 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerStarted","Data":"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557"} Jul 07 22:03:35.551656 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:35.551622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerStarted","Data":"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846"} Jul 07 22:03:35.552039 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:35.551682 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerStarted","Data":"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947"} Jul 07 22:03:35.552039 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:35.551709 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerStarted","Data":"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47"} Jul 07 22:03:35.552039 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:35.551738 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerStarted","Data":"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5"} Jul 07 22:03:35.580461 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:35.580411 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.725842414 podStartE2EDuration="7.58039454s" totalImagesPullingTime="3.821913558s" totalInitContainerRuntime="32.638568ms" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:35.57804225 +0000 UTC m=+85.142140677" watchObservedRunningTime="2026-07-07 22:03:35.58039454 +0000 UTC m=+85.144492966" Jul 07 22:03:36.533853 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:36.533827 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-74fc86b697-n572z" Jul 07 22:03:38.654330 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:38.654295 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:39.031733 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:39.031648 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:03:39.161579 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:39.161544 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:39.221796 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:39.221769 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:42.887438 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:42.887401 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fd9bf9464-zdvmf"] Jul 07 22:03:42.995800 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:42.995754 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:43.096681 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:43.096645 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:43.153543 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:43.153453 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:45.293770 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:45.293729 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:45.394149 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:45.394116 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:03:47.181200 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:47.181163 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:47.181789 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:47.181217 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:03:53.324368 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.324317 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6f8449ff65-tlx5p" podUID="3a83ace4-c925-432b-aefd-81ef71c51284" containerName="console" containerID="cri-o://9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14" gracePeriod=15 Jul 07 22:03:53.579753 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.579701 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:53.631492 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.631454 2580 generic.go:356] "Generic (PLEG): container finished" podID="3a83ace4-c925-432b-aefd-81ef71c51284" containerID="9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14" exitCode=0 Jul 07 22:03:53.631663 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.631501 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f8449ff65-tlx5p" Jul 07 22:03:53.631663 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.631498 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f8449ff65-tlx5p" event={"ID":"3a83ace4-c925-432b-aefd-81ef71c51284","Type":"ContainerDied","Data":"9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14"} Jul 07 22:03:53.631663 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.631600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f8449ff65-tlx5p" event={"ID":"3a83ace4-c925-432b-aefd-81ef71c51284","Type":"ContainerDied","Data":"3dd2c631506e40a8cecfcaced269417fe2b97e0543274575e2249d26b2b2bd14"} Jul 07 22:03:53.631663 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.631629 2580 scope.go:117] "RemoveContainer" containerID="9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14" Jul 07 22:03:53.640106 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.640086 2580 scope.go:117] "RemoveContainer" containerID="9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14" Jul 07 22:03:53.640374 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:53.640355 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14\": container with ID starting with 9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14 not found: ID does not exist" containerID="9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14" Jul 07 22:03:53.640453 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.640385 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14"} err="failed to get container status \"9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14\": rpc error: code = NotFound desc = could not find container \"9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14\": container with ID starting with 9f0ddfaebcde35854ae5f9a3ce9d2a99b102246b3f1f0f0917705d9ff50c8a14 not found: ID does not exist" Jul 07 22:03:53.657714 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.657687 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg42k\" (UniqueName: \"kubernetes.io/projected/3a83ace4-c925-432b-aefd-81ef71c51284-kube-api-access-rg42k\") pod \"3a83ace4-c925-432b-aefd-81ef71c51284\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " Jul 07 22:03:53.657794 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.657737 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-oauth-serving-cert\") pod \"3a83ace4-c925-432b-aefd-81ef71c51284\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " Jul 07 22:03:53.657794 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.657756 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-oauth-config\") pod \"3a83ace4-c925-432b-aefd-81ef71c51284\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " Jul 07 22:03:53.657875 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.657793 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-service-ca\") pod \"3a83ace4-c925-432b-aefd-81ef71c51284\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " Jul 07 22:03:53.657875 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.657818 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-serving-cert\") pod \"3a83ace4-c925-432b-aefd-81ef71c51284\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " Jul 07 22:03:53.657875 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.657850 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-console-config\") pod \"3a83ace4-c925-432b-aefd-81ef71c51284\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " Jul 07 22:03:53.658031 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.657945 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-trusted-ca-bundle\") pod \"3a83ace4-c925-432b-aefd-81ef71c51284\" (UID: \"3a83ace4-c925-432b-aefd-81ef71c51284\") " Jul 07 22:03:53.658183 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.658088 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3a83ace4-c925-432b-aefd-81ef71c51284" (UID: "3a83ace4-c925-432b-aefd-81ef71c51284"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:03:53.658249 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.658186 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-service-ca" (OuterVolumeSpecName: "service-ca") pod "3a83ace4-c925-432b-aefd-81ef71c51284" (UID: "3a83ace4-c925-432b-aefd-81ef71c51284"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:03:53.658249 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.658216 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-console-config" (OuterVolumeSpecName: "console-config") pod "3a83ace4-c925-432b-aefd-81ef71c51284" (UID: "3a83ace4-c925-432b-aefd-81ef71c51284"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:03:53.658352 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.658270 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-oauth-serving-cert\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:53.658406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.658363 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-service-ca\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:53.658459 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.658429 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3a83ace4-c925-432b-aefd-81ef71c51284" (UID: "3a83ace4-c925-432b-aefd-81ef71c51284"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:03:53.660313 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.660291 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3a83ace4-c925-432b-aefd-81ef71c51284" (UID: "3a83ace4-c925-432b-aefd-81ef71c51284"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:03:53.660504 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.660481 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3a83ace4-c925-432b-aefd-81ef71c51284" (UID: "3a83ace4-c925-432b-aefd-81ef71c51284"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:03:53.660504 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.660487 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a83ace4-c925-432b-aefd-81ef71c51284-kube-api-access-rg42k" (OuterVolumeSpecName: "kube-api-access-rg42k") pod "3a83ace4-c925-432b-aefd-81ef71c51284" (UID: "3a83ace4-c925-432b-aefd-81ef71c51284"). InnerVolumeSpecName "kube-api-access-rg42k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:03:53.758918 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.758880 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg42k\" (UniqueName: \"kubernetes.io/projected/3a83ace4-c925-432b-aefd-81ef71c51284-kube-api-access-rg42k\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:53.758918 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.758913 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-oauth-config\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:53.758918 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.758926 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a83ace4-c925-432b-aefd-81ef71c51284-console-serving-cert\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:53.759115 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.758940 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-console-config\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:53.759115 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.758955 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a83ace4-c925-432b-aefd-81ef71c51284-trusted-ca-bundle\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:53.953996 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.953966 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f8449ff65-tlx5p"] Jul 07 22:03:53.958280 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:53.958258 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f8449ff65-tlx5p"] Jul 07 22:03:54.415361 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:54.415330 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qsxfk" Jul 07 22:03:54.959588 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:54.959555 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a83ace4-c925-432b-aefd-81ef71c51284" path="/var/lib/kubelet/pods/3a83ace4-c925-432b-aefd-81ef71c51284/volumes" Jul 07 22:03:57.390289 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.390244 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-695df9c75f-bjt2l" podUID="cc49ffba-aa22-4dfc-a061-dd24f3690c30" containerName="console" containerID="cri-o://6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e" gracePeriod=15 Jul 07 22:03:57.654492 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.654418 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:57.656549 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.656484 2580 generic.go:356] "Generic (PLEG): container finished" podID="cc49ffba-aa22-4dfc-a061-dd24f3690c30" containerID="6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e" exitCode=0 Jul 07 22:03:57.656549 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.656537 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695df9c75f-bjt2l" event={"ID":"cc49ffba-aa22-4dfc-a061-dd24f3690c30","Type":"ContainerDied","Data":"6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e"} Jul 07 22:03:57.656755 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.656600 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695df9c75f-bjt2l" Jul 07 22:03:57.656755 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.656626 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695df9c75f-bjt2l" event={"ID":"cc49ffba-aa22-4dfc-a061-dd24f3690c30","Type":"ContainerDied","Data":"1e804f3469e967db576c62e5cd908fb17857acf64dcc981f9c85d17724488073"} Jul 07 22:03:57.656755 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.656664 2580 scope.go:117] "RemoveContainer" containerID="6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e" Jul 07 22:03:57.664842 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.664817 2580 scope.go:117] "RemoveContainer" containerID="6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e" Jul 07 22:03:57.665083 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:03:57.665068 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e\": container with ID starting with 6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e not found: ID does not exist" containerID="6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e" Jul 07 22:03:57.665132 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.665093 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e"} err="failed to get container status \"6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e\": rpc error: code = NotFound desc = could not find container \"6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e\": container with ID starting with 6c355e887d9f7be65ad49d78da55e4ff3f3388b05962f38213e8b8d7daf4fb7e not found: ID does not exist" Jul 07 22:03:57.695838 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.695809 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-oauth-serving-cert\") pod \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " Jul 07 22:03:57.695993 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.695861 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-service-ca\") pod \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " Jul 07 22:03:57.695993 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.695902 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-serving-cert\") pod \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " Jul 07 22:03:57.695993 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.695927 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-config\") pod \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " Jul 07 22:03:57.695993 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.695977 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-oauth-config\") pod \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " Jul 07 22:03:57.696197 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.696045 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwd54\" (UniqueName: \"kubernetes.io/projected/cc49ffba-aa22-4dfc-a061-dd24f3690c30-kube-api-access-hwd54\") pod \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " Jul 07 22:03:57.696197 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.696084 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-trusted-ca-bundle\") pod \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\" (UID: \"cc49ffba-aa22-4dfc-a061-dd24f3690c30\") " Jul 07 22:03:57.696384 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.696260 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cc49ffba-aa22-4dfc-a061-dd24f3690c30" (UID: "cc49ffba-aa22-4dfc-a061-dd24f3690c30"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:03:57.696384 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.696368 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-config" (OuterVolumeSpecName: "console-config") pod "cc49ffba-aa22-4dfc-a061-dd24f3690c30" (UID: "cc49ffba-aa22-4dfc-a061-dd24f3690c30"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:03:57.696538 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.696357 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-service-ca" (OuterVolumeSpecName: "service-ca") pod "cc49ffba-aa22-4dfc-a061-dd24f3690c30" (UID: "cc49ffba-aa22-4dfc-a061-dd24f3690c30"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:03:57.696538 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.696499 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cc49ffba-aa22-4dfc-a061-dd24f3690c30" (UID: "cc49ffba-aa22-4dfc-a061-dd24f3690c30"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:03:57.698306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.698277 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cc49ffba-aa22-4dfc-a061-dd24f3690c30" (UID: "cc49ffba-aa22-4dfc-a061-dd24f3690c30"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:03:57.698424 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.698401 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cc49ffba-aa22-4dfc-a061-dd24f3690c30" (UID: "cc49ffba-aa22-4dfc-a061-dd24f3690c30"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:03:57.698471 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.698415 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc49ffba-aa22-4dfc-a061-dd24f3690c30-kube-api-access-hwd54" (OuterVolumeSpecName: "kube-api-access-hwd54") pod "cc49ffba-aa22-4dfc-a061-dd24f3690c30" (UID: "cc49ffba-aa22-4dfc-a061-dd24f3690c30"). InnerVolumeSpecName "kube-api-access-hwd54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:03:57.797311 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.797277 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-serving-cert\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:57.797311 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.797308 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-config\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:57.797311 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.797318 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc49ffba-aa22-4dfc-a061-dd24f3690c30-console-oauth-config\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:57.797566 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.797327 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwd54\" (UniqueName: \"kubernetes.io/projected/cc49ffba-aa22-4dfc-a061-dd24f3690c30-kube-api-access-hwd54\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:57.797566 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.797336 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-trusted-ca-bundle\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:57.797566 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.797344 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-oauth-serving-cert\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:57.797566 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.797352 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc49ffba-aa22-4dfc-a061-dd24f3690c30-service-ca\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:03:57.978421 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.978395 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695df9c75f-bjt2l"] Jul 07 22:03:57.985595 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:57.985572 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-695df9c75f-bjt2l"] Jul 07 22:03:58.959604 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:03:58.959563 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc49ffba-aa22-4dfc-a061-dd24f3690c30" path="/var/lib/kubelet/pods/cc49ffba-aa22-4dfc-a061-dd24f3690c30/volumes" Jul 07 22:04:00.303140 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:00.303080 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:00.375709 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:00.375659 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:04.669199 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:04.669156 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:04.723444 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:04.723415 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:07.186748 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:07.186722 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:04:07.190548 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:07.190510 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-686d5bf558-vkqch" Jul 07 22:04:07.911559 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:07.911480 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5fd9bf9464-zdvmf" podUID="3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" containerName="console" containerID="cri-o://eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab" gracePeriod=15 Jul 07 22:04:08.170795 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.170727 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:04:08.292204 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.292172 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-oauth-config\") pod \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " Jul 07 22:04:08.292685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.292231 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-config\") pod \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " Jul 07 22:04:08.292685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.292261 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4hp4\" (UniqueName: \"kubernetes.io/projected/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-kube-api-access-h4hp4\") pod \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " Jul 07 22:04:08.292685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.292325 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-service-ca\") pod \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " Jul 07 22:04:08.292685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.292374 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-serving-cert\") pod \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " Jul 07 22:04:08.292685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.292397 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-oauth-serving-cert\") pod \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\" (UID: \"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b\") " Jul 07 22:04:08.292685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.292616 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-config" (OuterVolumeSpecName: "console-config") pod "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" (UID: "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:08.292972 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.292754 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-service-ca" (OuterVolumeSpecName: "service-ca") pod "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" (UID: "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:08.292972 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.292786 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-config\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:08.292972 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.292914 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" (UID: "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:08.295079 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.295025 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-kube-api-access-h4hp4" (OuterVolumeSpecName: "kube-api-access-h4hp4") pod "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" (UID: "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b"). InnerVolumeSpecName "kube-api-access-h4hp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:04:08.295079 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.295025 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" (UID: "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:08.295534 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.295489 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" (UID: "3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:08.393728 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.393699 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-oauth-config\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:08.393728 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.393724 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4hp4\" (UniqueName: \"kubernetes.io/projected/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-kube-api-access-h4hp4\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:08.393728 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.393734 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-service-ca\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:08.393920 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.393743 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-console-serving-cert\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:08.393920 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.393751 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b-oauth-serving-cert\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:08.703650 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.703613 2580 generic.go:356] "Generic (PLEG): container finished" podID="3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" containerID="eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab" exitCode=0 Jul 07 22:04:08.703815 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.703648 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fd9bf9464-zdvmf" event={"ID":"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b","Type":"ContainerDied","Data":"eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab"} Jul 07 22:04:08.703815 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.703670 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fd9bf9464-zdvmf" Jul 07 22:04:08.703815 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.703728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fd9bf9464-zdvmf" event={"ID":"3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b","Type":"ContainerDied","Data":"431fa7d07e6ead32b9974e4e5750d7efa8b9948bed2af32d056a511ecebbcc9e"} Jul 07 22:04:08.703815 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.703749 2580 scope.go:117] "RemoveContainer" containerID="eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab" Jul 07 22:04:08.712438 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.712421 2580 scope.go:117] "RemoveContainer" containerID="eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab" Jul 07 22:04:08.712715 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:08.712698 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab\": container with ID starting with eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab not found: ID does not exist" containerID="eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab" Jul 07 22:04:08.712790 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.712724 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab"} err="failed to get container status \"eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab\": rpc error: code = NotFound desc = could not find container \"eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab\": container with ID starting with eded7bc75f54b1623a0350624ac56380b3c888dd4dc837daacd78e6c958367ab not found: ID does not exist" Jul 07 22:04:08.725530 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.725486 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fd9bf9464-zdvmf"] Jul 07 22:04:08.729954 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.729932 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fd9bf9464-zdvmf"] Jul 07 22:04:08.958690 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:08.958607 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" path="/var/lib/kubelet/pods/3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b/volumes" Jul 07 22:04:25.664359 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:25.664323 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:29.031706 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:29.031669 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:29.051207 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:29.051175 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:29.804086 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:29.804058 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:35.112024 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:35.111984 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:38.559215 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:38.559183 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:38.619235 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:38.619205 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:40.576426 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:40.576396 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:40.633014 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:40.632987 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:42.859570 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:42.859537 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 22:04:42.859975 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:42.859855 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="alertmanager" containerID="cri-o://332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b" gracePeriod=120 Jul 07 22:04:42.859975 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:42.859878 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="prom-label-proxy" containerID="cri-o://9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd" gracePeriod=120 Jul 07 22:04:42.859975 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:42.859929 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy-metric" containerID="cri-o://b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d" gracePeriod=120 Jul 07 22:04:42.859975 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:42.859912 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy-web" containerID="cri-o://41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350" gracePeriod=120 Jul 07 22:04:42.860190 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:42.859878 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy" containerID="cri-o://c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77" gracePeriod=120 Jul 07 22:04:42.860190 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:42.859917 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="config-reloader" containerID="cri-o://3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19" gracePeriod=120 Jul 07 22:04:43.847216 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:43.847187 2580 generic.go:356] "Generic (PLEG): container finished" podID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerID="9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd" exitCode=0 Jul 07 22:04:43.847216 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:43.847212 2580 generic.go:356] "Generic (PLEG): container finished" podID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerID="c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77" exitCode=0 Jul 07 22:04:43.847216 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:43.847218 2580 generic.go:356] "Generic (PLEG): container finished" podID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerID="3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19" exitCode=0 Jul 07 22:04:43.847216 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:43.847225 2580 generic.go:356] "Generic (PLEG): container finished" podID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerID="332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b" exitCode=0 Jul 07 22:04:43.847472 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:43.847260 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerDied","Data":"9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd"} Jul 07 22:04:43.847472 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:43.847327 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerDied","Data":"c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77"} Jul 07 22:04:43.847472 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:43.847353 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerDied","Data":"3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19"} Jul 07 22:04:43.847472 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:43.847377 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerDied","Data":"332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b"} Jul 07 22:04:44.607265 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.607243 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:44.697381 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697355 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-main-db\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697567 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697392 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-main-tls\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697567 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697416 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-trusted-ca-bundle\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697567 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697447 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-out\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697567 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697470 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-web-config\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697567 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697492 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-volume\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697567 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697599 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-tls-assets\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697642 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697678 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-web\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697707 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4jpl\" (UniqueName: \"kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-kube-api-access-n4jpl\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697740 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:04:44.697834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697770 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-metrics-client-ca\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.697834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697794 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-cluster-tls-config\") pod \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\" (UID: \"e002aec2-e4a5-4592-9f6c-692c0b7624a2\") " Jul 07 22:04:44.698220 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.697853 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:44.698220 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.698038 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-main-db\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.698220 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.698057 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.698425 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.698388 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:44.700477 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.700440 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-volume" (OuterVolumeSpecName: "config-volume") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:44.700633 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.700502 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:44.700633 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.700600 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-out" (OuterVolumeSpecName: "config-out") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:04:44.700849 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.700819 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-kube-api-access-n4jpl" (OuterVolumeSpecName: "kube-api-access-n4jpl") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "kube-api-access-n4jpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:04:44.700973 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.700942 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:44.701616 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.701594 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:44.702964 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.702944 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:04:44.703032 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.702964 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:44.706447 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.706421 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:44.713324 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.713303 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-web-config" (OuterVolumeSpecName: "web-config") pod "e002aec2-e4a5-4592-9f6c-692c0b7624a2" (UID: "e002aec2-e4a5-4592-9f6c-692c0b7624a2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:44.798878 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.798812 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.798878 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.798847 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-tls-assets\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.798878 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.798863 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.799034 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.798881 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.799034 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.798896 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4jpl\" (UniqueName: \"kubernetes.io/projected/e002aec2-e4a5-4592-9f6c-692c0b7624a2-kube-api-access-n4jpl\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.799034 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.798910 2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e002aec2-e4a5-4592-9f6c-692c0b7624a2-metrics-client-ca\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.799034 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.798924 2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-cluster-tls-config\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.799034 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.798938 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-secret-alertmanager-main-tls\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.799034 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.798951 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-out\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.799034 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.798963 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-web-config\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.799034 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.798975 2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e002aec2-e4a5-4592-9f6c-692c0b7624a2-config-volume\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:44.854861 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.854833 2580 generic.go:356] "Generic (PLEG): container finished" podID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerID="b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d" exitCode=0 Jul 07 22:04:44.854861 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.854855 2580 generic.go:356] "Generic (PLEG): container finished" podID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerID="41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350" exitCode=0 Jul 07 22:04:44.855017 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.854909 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:44.855017 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.854907 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerDied","Data":"b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d"} Jul 07 22:04:44.855017 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.854975 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerDied","Data":"41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350"} Jul 07 22:04:44.855017 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.855005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e002aec2-e4a5-4592-9f6c-692c0b7624a2","Type":"ContainerDied","Data":"d1f542a46ce27cac3a10da4c35b4ef7a952c9445439281c14431b6133aed6119"} Jul 07 22:04:44.855146 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.855033 2580 scope.go:117] "RemoveContainer" containerID="9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd" Jul 07 22:04:44.865305 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.864563 2580 scope.go:117] "RemoveContainer" containerID="b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d" Jul 07 22:04:44.873416 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.873395 2580 scope.go:117] "RemoveContainer" containerID="c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77" Jul 07 22:04:44.880480 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.880459 2580 scope.go:117] "RemoveContainer" containerID="41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350" Jul 07 22:04:44.884656 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.884632 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 22:04:44.888064 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.888049 2580 scope.go:117] "RemoveContainer" containerID="3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19" Jul 07 22:04:44.894960 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.894939 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 22:04:44.896116 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.896103 2580 scope.go:117] "RemoveContainer" containerID="332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b" Jul 07 22:04:44.903282 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.903262 2580 scope.go:117] "RemoveContainer" containerID="04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616" Jul 07 22:04:44.910031 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.910016 2580 scope.go:117] "RemoveContainer" containerID="9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd" Jul 07 22:04:44.910263 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:44.910247 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd\": container with ID starting with 9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd not found: ID does not exist" containerID="9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd" Jul 07 22:04:44.910308 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.910273 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd"} err="failed to get container status \"9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd\": rpc error: code = NotFound desc = could not find container \"9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd\": container with ID starting with 9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd not found: ID does not exist" Jul 07 22:04:44.910308 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.910297 2580 scope.go:117] "RemoveContainer" containerID="b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d" Jul 07 22:04:44.910511 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:44.910491 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d\": container with ID starting with b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d not found: ID does not exist" containerID="b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d" Jul 07 22:04:44.910567 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.910538 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d"} err="failed to get container status \"b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d\": rpc error: code = NotFound desc = could not find container \"b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d\": container with ID starting with b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d not found: ID does not exist" Jul 07 22:04:44.910567 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.910555 2580 scope.go:117] "RemoveContainer" containerID="c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77" Jul 07 22:04:44.910751 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:44.910736 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77\": container with ID starting with c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77 not found: ID does not exist" containerID="c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77" Jul 07 22:04:44.910797 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.910755 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77"} err="failed to get container status \"c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77\": rpc error: code = NotFound desc = could not find container \"c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77\": container with ID starting with c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77 not found: ID does not exist" Jul 07 22:04:44.910797 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.910776 2580 scope.go:117] "RemoveContainer" containerID="41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350" Jul 07 22:04:44.910957 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:44.910943 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350\": container with ID starting with 41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350 not found: ID does not exist" containerID="41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350" Jul 07 22:04:44.910997 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.910960 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350"} err="failed to get container status \"41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350\": rpc error: code = NotFound desc = could not find container \"41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350\": container with ID starting with 41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350 not found: ID does not exist" Jul 07 22:04:44.910997 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.910972 2580 scope.go:117] "RemoveContainer" containerID="3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19" Jul 07 22:04:44.911157 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:44.911141 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19\": container with ID starting with 3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19 not found: ID does not exist" containerID="3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19" Jul 07 22:04:44.911198 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.911161 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19"} err="failed to get container status \"3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19\": rpc error: code = NotFound desc = could not find container \"3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19\": container with ID starting with 3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19 not found: ID does not exist" Jul 07 22:04:44.911198 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.911173 2580 scope.go:117] "RemoveContainer" containerID="332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b" Jul 07 22:04:44.911341 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:44.911327 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b\": container with ID starting with 332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b not found: ID does not exist" containerID="332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b" Jul 07 22:04:44.911377 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.911348 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b"} err="failed to get container status \"332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b\": rpc error: code = NotFound desc = could not find container \"332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b\": container with ID starting with 332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b not found: ID does not exist" Jul 07 22:04:44.911377 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.911364 2580 scope.go:117] "RemoveContainer" containerID="04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616" Jul 07 22:04:44.911621 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:44.911606 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616\": container with ID starting with 04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616 not found: ID does not exist" containerID="04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616" Jul 07 22:04:44.911670 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.911624 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616"} err="failed to get container status \"04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616\": rpc error: code = NotFound desc = could not find container \"04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616\": container with ID starting with 04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616 not found: ID does not exist" Jul 07 22:04:44.911670 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.911636 2580 scope.go:117] "RemoveContainer" containerID="9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd" Jul 07 22:04:44.911836 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.911822 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd"} err="failed to get container status \"9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd\": rpc error: code = NotFound desc = could not find container \"9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd\": container with ID starting with 9754ffa9740c48439c86fc4d8447181cd9eaf4abaab41837a505f3f881b1f5dd not found: ID does not exist" Jul 07 22:04:44.911875 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.911837 2580 scope.go:117] "RemoveContainer" containerID="b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d" Jul 07 22:04:44.912036 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.912020 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d"} err="failed to get container status \"b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d\": rpc error: code = NotFound desc = could not find container \"b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d\": container with ID starting with b558d118c41f097c17185f16bc5bd4bb409895c2a0eb33279f0b189d9ca4ed4d not found: ID does not exist" Jul 07 22:04:44.912077 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.912036 2580 scope.go:117] "RemoveContainer" containerID="c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77" Jul 07 22:04:44.912216 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.912201 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77"} err="failed to get container status \"c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77\": rpc error: code = NotFound desc = could not find container \"c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77\": container with ID starting with c9a1857a9222dd9985f8e9ae61df39418b293be0f79c706dee1768a8a2167a77 not found: ID does not exist" Jul 07 22:04:44.912256 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.912217 2580 scope.go:117] "RemoveContainer" containerID="41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350" Jul 07 22:04:44.912386 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.912372 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350"} err="failed to get container status \"41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350\": rpc error: code = NotFound desc = could not find container \"41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350\": container with ID starting with 41739c017e7b47e5551b72277305c77edc0a548fcf677d4918d2ed0335577350 not found: ID does not exist" Jul 07 22:04:44.912431 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.912386 2580 scope.go:117] "RemoveContainer" containerID="3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19" Jul 07 22:04:44.912583 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.912564 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19"} err="failed to get container status \"3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19\": rpc error: code = NotFound desc = could not find container \"3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19\": container with ID starting with 3cff471afb57b9cb3187860d74f8869a44245c2f58c043f851377df3f94ffe19 not found: ID does not exist" Jul 07 22:04:44.912673 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.912585 2580 scope.go:117] "RemoveContainer" containerID="332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b" Jul 07 22:04:44.912788 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.912773 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b"} err="failed to get container status \"332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b\": rpc error: code = NotFound desc = could not find container \"332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b\": container with ID starting with 332b6d6cf43f36ad864bfdd602b8f75b77d69adc606d9e1e863e06067de7c71b not found: ID does not exist" Jul 07 22:04:44.912833 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.912790 2580 scope.go:117] "RemoveContainer" containerID="04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616" Jul 07 22:04:44.912959 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.912944 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616"} err="failed to get container status \"04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616\": rpc error: code = NotFound desc = could not find container \"04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616\": container with ID starting with 04d849133af6e45d23f628adcb4860c355ee97ccb0a230978756ad08165a8616 not found: ID does not exist" Jul 07 22:04:44.926170 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926149 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 22:04:44.926470 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926458 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="init-config-reloader" Jul 07 22:04:44.926511 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926472 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="init-config-reloader" Jul 07 22:04:44.926511 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926481 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy-web" Jul 07 22:04:44.926511 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926486 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy-web" Jul 07 22:04:44.926511 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926493 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="prom-label-proxy" Jul 07 22:04:44.926511 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926499 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="prom-label-proxy" Jul 07 22:04:44.926511 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926506 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy-metric" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926512 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy-metric" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926548 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="config-reloader" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926553 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="config-reloader" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926559 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926564 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926572 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="alertmanager" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926576 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="alertmanager" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926583 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a83ace4-c925-432b-aefd-81ef71c51284" containerName="console" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926588 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a83ace4-c925-432b-aefd-81ef71c51284" containerName="console" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926595 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" containerName="console" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926600 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" containerName="console" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926605 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc49ffba-aa22-4dfc-a061-dd24f3690c30" containerName="console" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926610 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc49ffba-aa22-4dfc-a061-dd24f3690c30" containerName="console" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926656 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926664 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a83ace4-c925-432b-aefd-81ef71c51284" containerName="console" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926670 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy-metric" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926675 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc49ffba-aa22-4dfc-a061-dd24f3690c30" containerName="console" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926680 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="config-reloader" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926685 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="alertmanager" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926690 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="kube-rbac-proxy-web" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926696 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" containerName="prom-label-proxy" Jul 07 22:04:44.926696 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.926702 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3edf1e-8f24-4bd1-bb51-ff81e95f4b2b" containerName="console" Jul 07 22:04:44.930622 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.930605 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:44.932809 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.932792 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 22:04:44.932932 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.932859 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 22:04:44.932995 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.932936 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 22:04:44.932995 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.932974 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6j8h4\"" Jul 07 22:04:44.933117 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.933001 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 22:04:44.933117 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.933074 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 22:04:44.933208 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.933122 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 22:04:44.933208 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.933158 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 22:04:44.933454 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.933439 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 22:04:44.938248 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.938228 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 22:04:44.944875 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.944855 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 22:04:44.959836 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:44.959816 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e002aec2-e4a5-4592-9f6c-692c0b7624a2" path="/var/lib/kubelet/pods/e002aec2-e4a5-4592-9f6c-692c0b7624a2/volumes" Jul 07 22:04:45.000892 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.000862 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001059 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.000898 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82bd722a-8459-477c-95d9-a614d70bc24b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001059 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.000918 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82bd722a-8459-477c-95d9-a614d70bc24b-config-out\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001059 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.000939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001059 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.000975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn6rw\" (UniqueName: \"kubernetes.io/projected/82bd722a-8459-477c-95d9-a614d70bc24b-kube-api-access-nn6rw\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001059 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.001012 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001220 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.001153 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001220 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.001181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82bd722a-8459-477c-95d9-a614d70bc24b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001220 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.001200 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82bd722a-8459-477c-95d9-a614d70bc24b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001313 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.001217 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001313 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.001252 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-config-volume\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001373 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.001309 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82bd722a-8459-477c-95d9-a614d70bc24b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.001407 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.001372 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-web-config\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.101700 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.101819 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82bd722a-8459-477c-95d9-a614d70bc24b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.101819 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101725 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82bd722a-8459-477c-95d9-a614d70bc24b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.101819 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101742 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.101819 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-config-volume\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.101819 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101785 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82bd722a-8459-477c-95d9-a614d70bc24b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.101819 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101806 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-web-config\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.102091 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101835 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.102091 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101862 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82bd722a-8459-477c-95d9-a614d70bc24b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.102091 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82bd722a-8459-477c-95d9-a614d70bc24b-config-out\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.102091 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101928 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.102091 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101955 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6rw\" (UniqueName: \"kubernetes.io/projected/82bd722a-8459-477c-95d9-a614d70bc24b-kube-api-access-nn6rw\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.102091 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.101999 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.102391 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.102122 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82bd722a-8459-477c-95d9-a614d70bc24b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.102974 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.102948 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82bd722a-8459-477c-95d9-a614d70bc24b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.103582 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.103558 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82bd722a-8459-477c-95d9-a614d70bc24b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.105150 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.104820 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-config-volume\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.105150 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.104962 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82bd722a-8459-477c-95d9-a614d70bc24b-config-out\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.105150 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.104961 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.105150 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.105114 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.105398 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.105255 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.105398 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.105276 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.106155 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.106132 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82bd722a-8459-477c-95d9-a614d70bc24b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.106371 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.106353 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.107043 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.107025 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82bd722a-8459-477c-95d9-a614d70bc24b-web-config\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.113906 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.113883 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn6rw\" (UniqueName: \"kubernetes.io/projected/82bd722a-8459-477c-95d9-a614d70bc24b-kube-api-access-nn6rw\") pod \"alertmanager-main-0\" (UID: \"82bd722a-8459-477c-95d9-a614d70bc24b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.239826 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.239801 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 22:04:45.374070 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.374043 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 22:04:45.376467 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:04:45.376439 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82bd722a_8459_477c_95d9_a614d70bc24b.slice/crio-a65942c0c06926b6acd060f22e7f4ccc8d4bdd93f8d3767b1944e5bb3fc4b019 WatchSource:0}: Error finding container a65942c0c06926b6acd060f22e7f4ccc8d4bdd93f8d3767b1944e5bb3fc4b019: Status 404 returned error can't find the container with id a65942c0c06926b6acd060f22e7f4ccc8d4bdd93f8d3767b1944e5bb3fc4b019 Jul 07 22:04:45.860778 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.860743 2580 generic.go:356] "Generic (PLEG): container finished" podID="82bd722a-8459-477c-95d9-a614d70bc24b" containerID="97e2fb5b333e8354fbaffaaa8c181ef61625692a1709676855af76f29a49cc14" exitCode=0 Jul 07 22:04:45.861132 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.860829 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82bd722a-8459-477c-95d9-a614d70bc24b","Type":"ContainerDied","Data":"97e2fb5b333e8354fbaffaaa8c181ef61625692a1709676855af76f29a49cc14"} Jul 07 22:04:45.861132 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:45.860891 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82bd722a-8459-477c-95d9-a614d70bc24b","Type":"ContainerStarted","Data":"a65942c0c06926b6acd060f22e7f4ccc8d4bdd93f8d3767b1944e5bb3fc4b019"} Jul 07 22:04:46.868141 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:46.868106 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82bd722a-8459-477c-95d9-a614d70bc24b","Type":"ContainerStarted","Data":"be2015083a3eed5df648fdb4ed7f508c8f5ad9473c6d266b472aabeea2953561"} Jul 07 22:04:46.868509 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:46.868197 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82bd722a-8459-477c-95d9-a614d70bc24b","Type":"ContainerStarted","Data":"b0882c611f653b47500ef71451f789bb452f510dab0497d5f2655c23cfb1793d"} Jul 07 22:04:46.868509 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:46.868223 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82bd722a-8459-477c-95d9-a614d70bc24b","Type":"ContainerStarted","Data":"a9380a5c71dc7380d8ec10e44ff0395f721f99dc1627aa68bc87feb5c2a8d4ee"} Jul 07 22:04:46.868509 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:46.868247 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82bd722a-8459-477c-95d9-a614d70bc24b","Type":"ContainerStarted","Data":"c7fe91f624f38d53bea3dd432fea429612c50ac82ee046da39363d8c482c3012"} Jul 07 22:04:46.868509 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:46.868271 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82bd722a-8459-477c-95d9-a614d70bc24b","Type":"ContainerStarted","Data":"1db88e25f65811e09d19dcf2769ebed68d82e0cd9638d406969bff52c4a4c656"} Jul 07 22:04:46.868509 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:46.868294 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82bd722a-8459-477c-95d9-a614d70bc24b","Type":"ContainerStarted","Data":"7035bc05250e73c8789822b221d816b7aade22815ab742a7ef015252cff742e7"} Jul 07 22:04:47.147329 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.147178 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.133512542 podStartE2EDuration="3.147154684s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.642142ms" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:46.937055356 +0000 UTC m=+156.501153783" watchObservedRunningTime="2026-07-07 22:04:47.147154684 +0000 UTC m=+156.711253102" Jul 07 22:04:47.148443 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.148403 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:04:47.148748 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.148720 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="prometheus" containerID="cri-o://f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557" gracePeriod=600 Jul 07 22:04:47.148842 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.148754 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy" containerID="cri-o://c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947" gracePeriod=600 Jul 07 22:04:47.148842 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.148782 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846" gracePeriod=600 Jul 07 22:04:47.148842 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.148803 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="thanos-sidecar" containerID="cri-o://b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5" gracePeriod=600 Jul 07 22:04:47.149024 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.148876 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy-web" containerID="cri-o://5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47" gracePeriod=600 Jul 07 22:04:47.149024 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.148811 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="config-reloader" containerID="cri-o://d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1" gracePeriod=600 Jul 07 22:04:47.509054 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.509029 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:47.625452 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.625420 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-kube-rbac-proxy\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.625656 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.625463 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.625656 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.625489 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-serving-certs-ca-bundle\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.625656 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.625536 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-kubelet-serving-ca-bundle\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.625656 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.625561 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-grpc-tls\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.625656 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.625590 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-metrics-client-certs\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.625656 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.625623 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-tls\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.625951 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.625907 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-2\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.625951 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.625906 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:47.625951 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.625950 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-0\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626114 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626032 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:47.626167 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626153 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-trusted-ca-bundle\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626222 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626201 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-db\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626444 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626235 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config-out\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626500 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626441 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k66ks\" (UniqueName: \"kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-kube-api-access-k66ks\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626500 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626432 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:47.626500 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626477 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-metrics-client-ca\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626684 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626512 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626684 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626575 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626684 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626607 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-web-config\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626684 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626638 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-1\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626684 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626668 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-thanos-prometheus-http-client-file\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626943 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626704 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-tls-assets\") pod \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\" (UID: \"af5e29f1-0e94-4d25-808e-63b47a5bc85d\") " Jul 07 22:04:47.626995 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626954 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.626995 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626972 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.626995 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.626987 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.627829 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.627682 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:47.628105 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.628077 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:47.628176 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.628133 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:47.628176 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.628158 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:47.628554 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.628507 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:04:47.628790 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.628764 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:47.628923 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.628877 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:47.628923 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.628915 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:04:47.629316 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.629264 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:47.629651 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.629626 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config-out" (OuterVolumeSpecName: "config-out") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:04:47.630345 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.630315 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:47.630549 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.630530 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:04:47.630716 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.630692 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:47.630906 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.630887 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-kube-api-access-k66ks" (OuterVolumeSpecName: "kube-api-access-k66ks") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "kube-api-access-k66ks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:04:47.630963 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.630920 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:47.631044 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.631028 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config" (OuterVolumeSpecName: "config") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:47.641589 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.641568 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-web-config" (OuterVolumeSpecName: "web-config") pod "af5e29f1-0e94-4d25-808e-63b47a5bc85d" (UID: "af5e29f1-0e94-4d25-808e-63b47a5bc85d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:04:47.727685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727620 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727648 2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-grpc-tls\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727662 2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-metrics-client-certs\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727685 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727674 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727687 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727700 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727711 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-db\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727723 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config-out\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727736 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k66ks\" (UniqueName: \"kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-kube-api-access-k66ks\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727748 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-configmap-metrics-client-ca\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727761 2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-config\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727774 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727787 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-web-config\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727800 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af5e29f1-0e94-4d25-808e-63b47a5bc85d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727814 2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727827 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af5e29f1-0e94-4d25-808e-63b47a5bc85d-tls-assets\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.727911 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.727840 2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af5e29f1-0e94-4d25-808e-63b47a5bc85d-secret-kube-rbac-proxy\") on node \"ip-10-0-129-68.ec2.internal\" DevicePath \"\"" Jul 07 22:04:47.876112 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876081 2580 generic.go:356] "Generic (PLEG): container finished" podID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerID="7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846" exitCode=0 Jul 07 22:04:47.876112 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876110 2580 generic.go:356] "Generic (PLEG): container finished" podID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerID="c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947" exitCode=0 Jul 07 22:04:47.876555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876120 2580 generic.go:356] "Generic (PLEG): container finished" podID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerID="5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47" exitCode=0 Jul 07 22:04:47.876555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876130 2580 generic.go:356] "Generic (PLEG): container finished" podID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerID="b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5" exitCode=0 Jul 07 22:04:47.876555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876140 2580 generic.go:356] "Generic (PLEG): container finished" podID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerID="d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1" exitCode=0 Jul 07 22:04:47.876555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876148 2580 generic.go:356] "Generic (PLEG): container finished" podID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerID="f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557" exitCode=0 Jul 07 22:04:47.876555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876174 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:47.876555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876168 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerDied","Data":"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846"} Jul 07 22:04:47.876555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876383 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerDied","Data":"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947"} Jul 07 22:04:47.876555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876436 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerDied","Data":"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47"} Jul 07 22:04:47.876555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876443 2580 scope.go:117] "RemoveContainer" containerID="7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846" Jul 07 22:04:47.876555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876474 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerDied","Data":"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5"} Jul 07 22:04:47.876555 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876535 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerDied","Data":"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1"} Jul 07 22:04:47.877019 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876578 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerDied","Data":"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557"} Jul 07 22:04:47.877019 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.876621 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af5e29f1-0e94-4d25-808e-63b47a5bc85d","Type":"ContainerDied","Data":"a0cc7bd272320c40652e7254241a056184e2b945c89d6cb1651be760b255ec17"} Jul 07 22:04:47.885086 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.885070 2580 scope.go:117] "RemoveContainer" containerID="c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947" Jul 07 22:04:47.892047 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.892031 2580 scope.go:117] "RemoveContainer" containerID="5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47" Jul 07 22:04:47.898808 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.898789 2580 scope.go:117] "RemoveContainer" containerID="b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5" Jul 07 22:04:47.900810 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.900790 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:04:47.906682 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.906663 2580 scope.go:117] "RemoveContainer" containerID="d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1" Jul 07 22:04:47.908410 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.908388 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:04:47.913653 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.913633 2580 scope.go:117] "RemoveContainer" containerID="f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557" Jul 07 22:04:47.920540 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.920507 2580 scope.go:117] "RemoveContainer" containerID="72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5" Jul 07 22:04:47.926873 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.926858 2580 scope.go:117] "RemoveContainer" containerID="7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846" Jul 07 22:04:47.927137 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:47.927117 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": container with ID starting with 7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846 not found: ID does not exist" containerID="7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846" Jul 07 22:04:47.927215 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.927151 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846"} err="failed to get container status \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": rpc error: code = NotFound desc = could not find container \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": container with ID starting with 7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846 not found: ID does not exist" Jul 07 22:04:47.927215 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.927174 2580 scope.go:117] "RemoveContainer" containerID="c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947" Jul 07 22:04:47.927430 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:47.927413 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": container with ID starting with c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947 not found: ID does not exist" containerID="c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947" Jul 07 22:04:47.927470 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.927436 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947"} err="failed to get container status \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": rpc error: code = NotFound desc = could not find container \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": container with ID starting with c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947 not found: ID does not exist" Jul 07 22:04:47.927470 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.927452 2580 scope.go:117] "RemoveContainer" containerID="5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47" Jul 07 22:04:47.927695 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:47.927678 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": container with ID starting with 5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47 not found: ID does not exist" containerID="5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47" Jul 07 22:04:47.927770 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.927702 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47"} err="failed to get container status \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": rpc error: code = NotFound desc = could not find container \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": container with ID starting with 5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47 not found: ID does not exist" Jul 07 22:04:47.927770 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.927723 2580 scope.go:117] "RemoveContainer" containerID="b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5" Jul 07 22:04:47.927965 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:47.927948 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": container with ID starting with b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5 not found: ID does not exist" containerID="b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5" Jul 07 22:04:47.928003 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.927970 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5"} err="failed to get container status \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": rpc error: code = NotFound desc = could not find container \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": container with ID starting with b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5 not found: ID does not exist" Jul 07 22:04:47.928003 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.927984 2580 scope.go:117] "RemoveContainer" containerID="d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1" Jul 07 22:04:47.928217 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:47.928198 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": container with ID starting with d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1 not found: ID does not exist" containerID="d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1" Jul 07 22:04:47.928276 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.928223 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1"} err="failed to get container status \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": rpc error: code = NotFound desc = could not find container \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": container with ID starting with d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1 not found: ID does not exist" Jul 07 22:04:47.928276 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.928237 2580 scope.go:117] "RemoveContainer" containerID="f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557" Jul 07 22:04:47.928449 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:47.928432 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": container with ID starting with f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557 not found: ID does not exist" containerID="f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557" Jul 07 22:04:47.928501 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.928453 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557"} err="failed to get container status \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": rpc error: code = NotFound desc = could not find container \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": container with ID starting with f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557 not found: ID does not exist" Jul 07 22:04:47.928501 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.928466 2580 scope.go:117] "RemoveContainer" containerID="72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5" Jul 07 22:04:47.928755 ip-10-0-129-68 kubenswrapper[2580]: E0707 22:04:47.928730 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": container with ID starting with 72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5 not found: ID does not exist" containerID="72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5" Jul 07 22:04:47.928851 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.928760 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5"} err="failed to get container status \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": rpc error: code = NotFound desc = could not find container \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": container with ID starting with 72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5 not found: ID does not exist" Jul 07 22:04:47.928851 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.928780 2580 scope.go:117] "RemoveContainer" containerID="7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846" Jul 07 22:04:47.929092 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.929063 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846"} err="failed to get container status \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": rpc error: code = NotFound desc = could not find container \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": container with ID starting with 7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846 not found: ID does not exist" Jul 07 22:04:47.929092 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.929090 2580 scope.go:117] "RemoveContainer" containerID="c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947" Jul 07 22:04:47.929396 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.929334 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947"} err="failed to get container status \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": rpc error: code = NotFound desc = could not find container \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": container with ID starting with c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947 not found: ID does not exist" Jul 07 22:04:47.929396 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.929362 2580 scope.go:117] "RemoveContainer" containerID="5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47" Jul 07 22:04:47.929702 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.929595 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47"} err="failed to get container status \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": rpc error: code = NotFound desc = could not find container \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": container with ID starting with 5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47 not found: ID does not exist" Jul 07 22:04:47.929702 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.929618 2580 scope.go:117] "RemoveContainer" containerID="b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5" Jul 07 22:04:47.930114 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.930095 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5"} err="failed to get container status \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": rpc error: code = NotFound desc = could not find container \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": container with ID starting with b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5 not found: ID does not exist" Jul 07 22:04:47.930114 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.930115 2580 scope.go:117] "RemoveContainer" containerID="d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1" Jul 07 22:04:47.930359 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.930328 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1"} err="failed to get container status \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": rpc error: code = NotFound desc = could not find container \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": container with ID starting with d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1 not found: ID does not exist" Jul 07 22:04:47.930419 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.930360 2580 scope.go:117] "RemoveContainer" containerID="f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557" Jul 07 22:04:47.930584 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.930564 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557"} err="failed to get container status \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": rpc error: code = NotFound desc = could not find container \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": container with ID starting with f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557 not found: ID does not exist" Jul 07 22:04:47.930630 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.930586 2580 scope.go:117] "RemoveContainer" containerID="72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5" Jul 07 22:04:47.930783 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.930765 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5"} err="failed to get container status \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": rpc error: code = NotFound desc = could not find container \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": container with ID starting with 72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5 not found: ID does not exist" Jul 07 22:04:47.930823 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.930785 2580 scope.go:117] "RemoveContainer" containerID="7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846" Jul 07 22:04:47.930975 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.930960 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846"} err="failed to get container status \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": rpc error: code = NotFound desc = could not find container \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": container with ID starting with 7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846 not found: ID does not exist" Jul 07 22:04:47.931024 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.930975 2580 scope.go:117] "RemoveContainer" containerID="c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947" Jul 07 22:04:47.931157 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.931142 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947"} err="failed to get container status \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": rpc error: code = NotFound desc = could not find container \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": container with ID starting with c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947 not found: ID does not exist" Jul 07 22:04:47.931204 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.931157 2580 scope.go:117] "RemoveContainer" containerID="5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47" Jul 07 22:04:47.931329 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.931314 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47"} err="failed to get container status \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": rpc error: code = NotFound desc = could not find container \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": container with ID starting with 5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47 not found: ID does not exist" Jul 07 22:04:47.931373 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.931330 2580 scope.go:117] "RemoveContainer" containerID="b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5" Jul 07 22:04:47.931507 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.931490 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5"} err="failed to get container status \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": rpc error: code = NotFound desc = could not find container \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": container with ID starting with b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5 not found: ID does not exist" Jul 07 22:04:47.931559 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.931507 2580 scope.go:117] "RemoveContainer" containerID="d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1" Jul 07 22:04:47.931725 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.931709 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1"} err="failed to get container status \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": rpc error: code = NotFound desc = could not find container \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": container with ID starting with d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1 not found: ID does not exist" Jul 07 22:04:47.931764 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.931725 2580 scope.go:117] "RemoveContainer" containerID="f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557" Jul 07 22:04:47.931903 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.931887 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557"} err="failed to get container status \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": rpc error: code = NotFound desc = could not find container \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": container with ID starting with f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557 not found: ID does not exist" Jul 07 22:04:47.931944 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.931903 2580 scope.go:117] "RemoveContainer" containerID="72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5" Jul 07 22:04:47.932121 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.932106 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5"} err="failed to get container status \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": rpc error: code = NotFound desc = could not find container \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": container with ID starting with 72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5 not found: ID does not exist" Jul 07 22:04:47.932161 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.932122 2580 scope.go:117] "RemoveContainer" containerID="7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846" Jul 07 22:04:47.932307 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.932293 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846"} err="failed to get container status \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": rpc error: code = NotFound desc = could not find container \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": container with ID starting with 7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846 not found: ID does not exist" Jul 07 22:04:47.932354 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.932306 2580 scope.go:117] "RemoveContainer" containerID="c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947" Jul 07 22:04:47.932487 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.932472 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947"} err="failed to get container status \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": rpc error: code = NotFound desc = could not find container \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": container with ID starting with c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947 not found: ID does not exist" Jul 07 22:04:47.932547 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.932487 2580 scope.go:117] "RemoveContainer" containerID="5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47" Jul 07 22:04:47.932750 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.932735 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47"} err="failed to get container status \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": rpc error: code = NotFound desc = could not find container \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": container with ID starting with 5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47 not found: ID does not exist" Jul 07 22:04:47.932793 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.932750 2580 scope.go:117] "RemoveContainer" containerID="b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5" Jul 07 22:04:47.932931 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.932916 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5"} err="failed to get container status \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": rpc error: code = NotFound desc = could not find container \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": container with ID starting with b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5 not found: ID does not exist" Jul 07 22:04:47.932970 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.932932 2580 scope.go:117] "RemoveContainer" containerID="d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1" Jul 07 22:04:47.933090 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.933076 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1"} err="failed to get container status \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": rpc error: code = NotFound desc = could not find container \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": container with ID starting with d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1 not found: ID does not exist" Jul 07 22:04:47.933136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.933091 2580 scope.go:117] "RemoveContainer" containerID="f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557" Jul 07 22:04:47.933288 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.933274 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557"} err="failed to get container status \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": rpc error: code = NotFound desc = could not find container \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": container with ID starting with f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557 not found: ID does not exist" Jul 07 22:04:47.933327 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.933288 2580 scope.go:117] "RemoveContainer" containerID="72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5" Jul 07 22:04:47.933482 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.933467 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5"} err="failed to get container status \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": rpc error: code = NotFound desc = could not find container \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": container with ID starting with 72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5 not found: ID does not exist" Jul 07 22:04:47.933538 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.933482 2580 scope.go:117] "RemoveContainer" containerID="7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846" Jul 07 22:04:47.933688 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.933673 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846"} err="failed to get container status \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": rpc error: code = NotFound desc = could not find container \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": container with ID starting with 7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846 not found: ID does not exist" Jul 07 22:04:47.933729 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.933688 2580 scope.go:117] "RemoveContainer" containerID="c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947" Jul 07 22:04:47.933883 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.933864 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947"} err="failed to get container status \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": rpc error: code = NotFound desc = could not find container \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": container with ID starting with c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947 not found: ID does not exist" Jul 07 22:04:47.933943 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.933885 2580 scope.go:117] "RemoveContainer" containerID="5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47" Jul 07 22:04:47.934081 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.934063 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47"} err="failed to get container status \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": rpc error: code = NotFound desc = could not find container \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": container with ID starting with 5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47 not found: ID does not exist" Jul 07 22:04:47.934151 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.934083 2580 scope.go:117] "RemoveContainer" containerID="b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5" Jul 07 22:04:47.934259 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.934241 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5"} err="failed to get container status \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": rpc error: code = NotFound desc = could not find container \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": container with ID starting with b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5 not found: ID does not exist" Jul 07 22:04:47.934313 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.934259 2580 scope.go:117] "RemoveContainer" containerID="d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1" Jul 07 22:04:47.934463 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.934446 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1"} err="failed to get container status \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": rpc error: code = NotFound desc = could not find container \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": container with ID starting with d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1 not found: ID does not exist" Jul 07 22:04:47.934538 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.934464 2580 scope.go:117] "RemoveContainer" containerID="f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557" Jul 07 22:04:47.934683 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.934666 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557"} err="failed to get container status \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": rpc error: code = NotFound desc = could not find container \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": container with ID starting with f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557 not found: ID does not exist" Jul 07 22:04:47.934737 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.934684 2580 scope.go:117] "RemoveContainer" containerID="72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5" Jul 07 22:04:47.934871 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.934854 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5"} err="failed to get container status \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": rpc error: code = NotFound desc = could not find container \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": container with ID starting with 72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5 not found: ID does not exist" Jul 07 22:04:47.934927 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.934871 2580 scope.go:117] "RemoveContainer" containerID="7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846" Jul 07 22:04:47.935053 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.935037 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846"} err="failed to get container status \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": rpc error: code = NotFound desc = could not find container \"7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846\": container with ID starting with 7c9f64dc67f607ac9a7141629d643399c777b3b85ebfdb315640e413725b4846 not found: ID does not exist" Jul 07 22:04:47.935096 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.935054 2580 scope.go:117] "RemoveContainer" containerID="c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947" Jul 07 22:04:47.935247 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.935232 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947"} err="failed to get container status \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": rpc error: code = NotFound desc = could not find container \"c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947\": container with ID starting with c70eff6b39e8f104ff1308cb020cf0a8af48dc32a7bf6618a9924411b0e4c947 not found: ID does not exist" Jul 07 22:04:47.935291 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.935248 2580 scope.go:117] "RemoveContainer" containerID="5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47" Jul 07 22:04:47.935417 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.935402 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47"} err="failed to get container status \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": rpc error: code = NotFound desc = could not find container \"5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47\": container with ID starting with 5ff7ca861cdd6da2f05c51168157a5f4f722a2ea0bccbe9ab4ed9340a504cc47 not found: ID does not exist" Jul 07 22:04:47.935455 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.935417 2580 scope.go:117] "RemoveContainer" containerID="b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5" Jul 07 22:04:47.935611 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.935596 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5"} err="failed to get container status \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": rpc error: code = NotFound desc = could not find container \"b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5\": container with ID starting with b88724a6f6aeaaf68426083eb1542481c5f5f2e35b7169b674f7166912cff9f5 not found: ID does not exist" Jul 07 22:04:47.935654 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.935614 2580 scope.go:117] "RemoveContainer" containerID="d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1" Jul 07 22:04:47.935796 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.935778 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1"} err="failed to get container status \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": rpc error: code = NotFound desc = could not find container \"d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1\": container with ID starting with d506a429c999028c1b2776b97fba379def7bfaab5d69607be0057ce8697a86a1 not found: ID does not exist" Jul 07 22:04:47.935869 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.935797 2580 scope.go:117] "RemoveContainer" containerID="f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557" Jul 07 22:04:47.936005 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.935987 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557"} err="failed to get container status \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": rpc error: code = NotFound desc = could not find container \"f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557\": container with ID starting with f658806b6a20fdece7f32df3e7fa6994d0f52e873fc664e5cb174edb8fa13557 not found: ID does not exist" Jul 07 22:04:47.936040 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.936005 2580 scope.go:117] "RemoveContainer" containerID="72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5" Jul 07 22:04:47.936228 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.936205 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5"} err="failed to get container status \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": rpc error: code = NotFound desc = could not find container \"72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5\": container with ID starting with 72c1b1fa36d6a9531b14fe6c54471120894646fd6e86eb19a21e1db566495ef5 not found: ID does not exist" Jul 07 22:04:47.940454 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940434 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:04:47.940767 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940752 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="init-config-reloader" Jul 07 22:04:47.940818 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940770 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="init-config-reloader" Jul 07 22:04:47.940818 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940782 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy-thanos" Jul 07 22:04:47.940818 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940787 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy-thanos" Jul 07 22:04:47.940818 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940797 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="thanos-sidecar" Jul 07 22:04:47.940818 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940803 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="thanos-sidecar" Jul 07 22:04:47.940818 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940817 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940822 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940834 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="config-reloader" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940843 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="config-reloader" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940852 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy-web" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940857 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy-web" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940865 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="prometheus" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940870 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="prometheus" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940909 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy-thanos" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940922 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940929 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="prometheus" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940933 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="kube-rbac-proxy-web" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940937 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="config-reloader" Jul 07 22:04:47.940983 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.940942 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" containerName="thanos-sidecar" Jul 07 22:04:47.944773 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.944759 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:47.948705 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.948686 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 22:04:47.948795 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.948686 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 22:04:47.948868 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.948851 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 22:04:47.948992 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.948976 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 22:04:47.949055 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.949012 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 22:04:47.949136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.949124 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 22:04:47.949267 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.949249 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 22:04:47.949359 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.949278 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 22:04:47.949359 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.949334 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 22:04:47.949359 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.949352 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 22:04:47.949510 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.949419 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 22:04:47.949510 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.949481 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 22:04:47.949510 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.949493 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k89n2\"" Jul 07 22:04:47.949510 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.949534 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f6vh8dpimejbh\"" Jul 07 22:04:47.958350 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.958329 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 22:04:47.958507 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.958490 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 22:04:47.968871 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:47.968847 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:04:48.029966 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.029893 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.029966 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.029923 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-web-config\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.029966 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.029946 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030004 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030088 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030136 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030103 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxg4d\" (UniqueName: \"kubernetes.io/projected/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-kube-api-access-vxg4d\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030278 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030179 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030278 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030210 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030278 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030231 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-config\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030278 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030278 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030269 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030414 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030299 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030414 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030319 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030414 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030350 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-config-out\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030471 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030581 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030501 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.030613 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.030602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131189 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131189 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131190 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131348 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131348 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131433 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxg4d\" (UniqueName: \"kubernetes.io/projected/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-kube-api-access-vxg4d\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131433 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131406 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131433 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131601 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-config\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131601 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131601 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131508 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131601 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131786 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131786 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131625 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131786 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131655 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-config-out\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131786 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131677 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131786 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131717 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.131786 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.132080 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.132080 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-web-config\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.132080 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131837 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.132080 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131848 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.132080 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.131951 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.132336 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.132312 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.132817 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.132790 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.134360 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.134053 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.134866 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.134491 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.134964 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.134924 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.135633 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.135048 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.135633 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.135108 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-config-out\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.135633 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.135305 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.135633 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.135554 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.135633 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.135609 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-config\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.135930 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.135861 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.136508 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.136476 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-web-config\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.136612 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.136592 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.136706 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.136690 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.137297 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.137281 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.137977 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.137961 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.138491 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.138473 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.140931 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.140913 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxg4d\" (UniqueName: \"kubernetes.io/projected/b3321abe-0f1c-4f6e-9be0-1b3c17e52847-kube-api-access-vxg4d\") pod \"prometheus-k8s-0\" (UID: \"b3321abe-0f1c-4f6e-9be0-1b3c17e52847\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.256729 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.256701 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:04:48.391678 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.391653 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:04:48.393157 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:04:48.393130 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3321abe_0f1c_4f6e_9be0_1b3c17e52847.slice/crio-3bdf959915a5347f177d2485473e4e437e6d252a2cb16012aae3721c945cf93c WatchSource:0}: Error finding container 3bdf959915a5347f177d2485473e4e437e6d252a2cb16012aae3721c945cf93c: Status 404 returned error can't find the container with id 3bdf959915a5347f177d2485473e4e437e6d252a2cb16012aae3721c945cf93c Jul 07 22:04:48.884548 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.884498 2580 generic.go:356] "Generic (PLEG): container finished" podID="b3321abe-0f1c-4f6e-9be0-1b3c17e52847" containerID="298f51c1868af9ee5b4c9380d3deb11dfd26f0519145730942e20e0aee260383" exitCode=0 Jul 07 22:04:48.884933 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.884567 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3321abe-0f1c-4f6e-9be0-1b3c17e52847","Type":"ContainerDied","Data":"298f51c1868af9ee5b4c9380d3deb11dfd26f0519145730942e20e0aee260383"} Jul 07 22:04:48.884933 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.884617 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3321abe-0f1c-4f6e-9be0-1b3c17e52847","Type":"ContainerStarted","Data":"3bdf959915a5347f177d2485473e4e437e6d252a2cb16012aae3721c945cf93c"} Jul 07 22:04:48.959661 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:48.959637 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af5e29f1-0e94-4d25-808e-63b47a5bc85d" path="/var/lib/kubelet/pods/af5e29f1-0e94-4d25-808e-63b47a5bc85d/volumes" Jul 07 22:04:49.892903 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:49.892859 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3321abe-0f1c-4f6e-9be0-1b3c17e52847","Type":"ContainerStarted","Data":"b61f22f5fbddc6933e7ef1f3f9e70825432ab25965be1b7dae387d76010f7e2f"} Jul 07 22:04:49.893287 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:49.892971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3321abe-0f1c-4f6e-9be0-1b3c17e52847","Type":"ContainerStarted","Data":"7534aef035277b5029f91794d277b7093c76eafbd054dd64b1e790e023e8febb"} Jul 07 22:04:49.893287 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:49.893012 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3321abe-0f1c-4f6e-9be0-1b3c17e52847","Type":"ContainerStarted","Data":"3e8256cc0ff23c414042c78cdb684ab65929727b8bf89911738c86c5d47af9ba"} Jul 07 22:04:49.893287 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:49.893051 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3321abe-0f1c-4f6e-9be0-1b3c17e52847","Type":"ContainerStarted","Data":"89fc9637a2d31058196c234b52d2730d8ee61265bdc97938ee7783942b62d1ef"} Jul 07 22:04:49.893287 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:49.893084 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3321abe-0f1c-4f6e-9be0-1b3c17e52847","Type":"ContainerStarted","Data":"305b222fa59ba7a6c67dfe9abc1b32702ab95041c905d19a701120083ce4dc5f"} Jul 07 22:04:49.893287 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:49.893119 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3321abe-0f1c-4f6e-9be0-1b3c17e52847","Type":"ContainerStarted","Data":"97b60f7adc5911c38b9da1b54e00d15d569b5bc9b48b5f1b21f3acf8caf385c1"} Jul 07 22:04:49.925058 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:49.925002 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.90331755 podStartE2EDuration="2.924980409s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.662859ms" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:49.922945621 +0000 UTC m=+159.487044050" watchObservedRunningTime="2026-07-07 22:04:49.924980409 +0000 UTC m=+159.489078837" Jul 07 22:04:50.462565 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:50.462502 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:50.522306 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:50.522278 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:51.224686 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:51.224655 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:51.325649 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:51.325604 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:51.431149 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:51.431120 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:52.368600 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:52.368496 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:04:53.257619 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:04:53.257575 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:00.299498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:05:00.299439 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:05:00.398129 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:05:00.398090 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:05:08.169453 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:05:08.169407 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:05:33.093965 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:05:33.093923 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:05:48.257459 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:05:48.257428 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:48.273363 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:05:48.273341 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:49.137186 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:05:49.137157 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:06:02.504937 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:02.504897 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:06:29.960360 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:29.960330 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sbz4h/must-gather-gnv69"] Jul 07 22:06:29.961754 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:29.961722 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:06:29.962834 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:29.962818 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sbz4h/must-gather-gnv69" Jul 07 22:06:29.965243 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:29.965224 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sbz4h\"/\"kube-root-ca.crt\"" Jul 07 22:06:29.965347 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:29.965227 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sbz4h\"/\"default-dockercfg-9lhtt\"" Jul 07 22:06:29.965894 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:29.965879 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sbz4h\"/\"openshift-service-ca.crt\"" Jul 07 22:06:29.971053 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:29.971025 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sbz4h/must-gather-gnv69"] Jul 07 22:06:30.061418 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:30.061393 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" limit=1024 Jul 07 22:06:30.097627 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:30.097590 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96thf\" (UniqueName: \"kubernetes.io/projected/cd9bea04-2bc9-4ce8-974f-73d0967f67d9-kube-api-access-96thf\") pod \"must-gather-gnv69\" (UID: \"cd9bea04-2bc9-4ce8-974f-73d0967f67d9\") " pod="openshift-must-gather-sbz4h/must-gather-gnv69" Jul 07 22:06:30.097708 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:30.097650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cd9bea04-2bc9-4ce8-974f-73d0967f67d9-must-gather-output\") pod \"must-gather-gnv69\" (UID: \"cd9bea04-2bc9-4ce8-974f-73d0967f67d9\") " pod="openshift-must-gather-sbz4h/must-gather-gnv69" Jul 07 22:06:30.198314 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:30.198288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96thf\" (UniqueName: \"kubernetes.io/projected/cd9bea04-2bc9-4ce8-974f-73d0967f67d9-kube-api-access-96thf\") pod \"must-gather-gnv69\" (UID: \"cd9bea04-2bc9-4ce8-974f-73d0967f67d9\") " pod="openshift-must-gather-sbz4h/must-gather-gnv69" Jul 07 22:06:30.198435 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:30.198324 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cd9bea04-2bc9-4ce8-974f-73d0967f67d9-must-gather-output\") pod \"must-gather-gnv69\" (UID: \"cd9bea04-2bc9-4ce8-974f-73d0967f67d9\") " pod="openshift-must-gather-sbz4h/must-gather-gnv69" Jul 07 22:06:30.198686 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:30.198669 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cd9bea04-2bc9-4ce8-974f-73d0967f67d9-must-gather-output\") pod \"must-gather-gnv69\" (UID: \"cd9bea04-2bc9-4ce8-974f-73d0967f67d9\") " pod="openshift-must-gather-sbz4h/must-gather-gnv69" Jul 07 22:06:30.207649 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:30.207626 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96thf\" (UniqueName: \"kubernetes.io/projected/cd9bea04-2bc9-4ce8-974f-73d0967f67d9-kube-api-access-96thf\") pod \"must-gather-gnv69\" (UID: \"cd9bea04-2bc9-4ce8-974f-73d0967f67d9\") " pod="openshift-must-gather-sbz4h/must-gather-gnv69" Jul 07 22:06:30.284937 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:30.284865 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sbz4h/must-gather-gnv69" Jul 07 22:06:30.413862 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:30.413837 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sbz4h/must-gather-gnv69"] Jul 07 22:06:30.416473 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:06:30.416439 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd9bea04_2bc9_4ce8_974f_73d0967f67d9.slice/crio-93768eed98e36f04613d73ca20644a2835f206dd9ffb92748c69a211f9cbaee2 WatchSource:0}: Error finding container 93768eed98e36f04613d73ca20644a2835f206dd9ffb92748c69a211f9cbaee2: Status 404 returned error can't find the container with id 93768eed98e36f04613d73ca20644a2835f206dd9ffb92748c69a211f9cbaee2 Jul 07 22:06:31.289489 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:31.289452 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sbz4h/must-gather-gnv69" event={"ID":"cd9bea04-2bc9-4ce8-974f-73d0967f67d9","Type":"ContainerStarted","Data":"93768eed98e36f04613d73ca20644a2835f206dd9ffb92748c69a211f9cbaee2"} Jul 07 22:06:32.295084 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:32.295053 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sbz4h/must-gather-gnv69" event={"ID":"cd9bea04-2bc9-4ce8-974f-73d0967f67d9","Type":"ContainerStarted","Data":"618f6145f28ecd9b4d9dcf93376625cae15227af3df2bd82b413ba56b7be58a7"} Jul 07 22:06:32.295468 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:32.295093 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sbz4h/must-gather-gnv69" event={"ID":"cd9bea04-2bc9-4ce8-974f-73d0967f67d9","Type":"ContainerStarted","Data":"c4dfd9dd7e32de9dfeff6691a851ce29693d5ed9f00a70f9032c5454c769768a"} Jul 07 22:06:32.315344 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:32.315277 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sbz4h/must-gather-gnv69" podStartSLOduration=2.402324727 podStartE2EDuration="3.315255658s" totalImagesPullingTime="912.930931ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:06:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:06:32.314659221 +0000 UTC m=+261.878757650" watchObservedRunningTime="2026-07-07 22:06:32.315255658 +0000 UTC m=+261.879354083" Jul 07 22:06:32.682568 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:32.682538 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-59xcs_60d1d68e-fcca-4c49-9d19-91608847c10b/global-pull-secret-syncer/0.log" Jul 07 22:06:32.836916 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:32.836886 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2hj4g_5c272d87-a60e-4d9f-b942-655bbcb9b788/konnectivity-agent/0.log" Jul 07 22:06:32.921116 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:32.921082 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-68.ec2.internal_2cc3ad308bf9860b5b3bfada611aed6f/haproxy/0.log" Jul 07 22:06:36.749026 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:36.748985 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82bd722a-8459-477c-95d9-a614d70bc24b/alertmanager/0.log" Jul 07 22:06:36.779928 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:36.779881 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82bd722a-8459-477c-95d9-a614d70bc24b/config-reloader/0.log" Jul 07 22:06:36.809203 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:36.809170 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82bd722a-8459-477c-95d9-a614d70bc24b/kube-rbac-proxy-web/0.log" Jul 07 22:06:36.839406 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:36.839362 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82bd722a-8459-477c-95d9-a614d70bc24b/kube-rbac-proxy/0.log" Jul 07 22:06:36.875728 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:36.875679 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82bd722a-8459-477c-95d9-a614d70bc24b/kube-rbac-proxy-metric/0.log" Jul 07 22:06:36.922498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:36.922413 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82bd722a-8459-477c-95d9-a614d70bc24b/prom-label-proxy/0.log" Jul 07 22:06:36.967809 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:36.967778 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82bd722a-8459-477c-95d9-a614d70bc24b/init-config-reloader/0.log" Jul 07 22:06:37.074035 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.074000 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-zf9cg_00b7c58f-c0ac-4597-a833-d9b7fc4aecd3/kube-state-metrics/0.log" Jul 07 22:06:37.114569 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.114541 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-zf9cg_00b7c58f-c0ac-4597-a833-d9b7fc4aecd3/kube-rbac-proxy-main/0.log" Jul 07 22:06:37.153924 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.153890 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-zf9cg_00b7c58f-c0ac-4597-a833-d9b7fc4aecd3/kube-rbac-proxy-self/0.log" Jul 07 22:06:37.196855 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.196771 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-686d5bf558-vkqch_62b999e7-9b38-42be-8c54-4cd8862944bc/metrics-server/0.log" Jul 07 22:06:37.245699 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.245662 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-5629p_2d1644a7-f5c5-4fc2-a33b-86bcb1f19cfe/monitoring-plugin/0.log" Jul 07 22:06:37.290107 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.290074 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6f2th_55eb3979-c8a2-4871-8dcb-938f02ee61e5/node-exporter/0.log" Jul 07 22:06:37.322569 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.322541 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6f2th_55eb3979-c8a2-4871-8dcb-938f02ee61e5/kube-rbac-proxy/0.log" Jul 07 22:06:37.363502 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.363473 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6f2th_55eb3979-c8a2-4871-8dcb-938f02ee61e5/init-textfile/0.log" Jul 07 22:06:37.653888 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.653858 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-kf2d4_81ad86f9-d1db-4503-aab2-44589cb920c5/kube-rbac-proxy-main/0.log" Jul 07 22:06:37.697854 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.697820 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-kf2d4_81ad86f9-d1db-4503-aab2-44589cb920c5/kube-rbac-proxy-self/0.log" Jul 07 22:06:37.739128 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.739095 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-kf2d4_81ad86f9-d1db-4503-aab2-44589cb920c5/openshift-state-metrics/0.log" Jul 07 22:06:37.822915 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.822838 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3321abe-0f1c-4f6e-9be0-1b3c17e52847/prometheus/0.log" Jul 07 22:06:37.856197 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.856170 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3321abe-0f1c-4f6e-9be0-1b3c17e52847/config-reloader/0.log" Jul 07 22:06:37.907779 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.907751 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3321abe-0f1c-4f6e-9be0-1b3c17e52847/thanos-sidecar/0.log" Jul 07 22:06:37.944201 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.944171 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3321abe-0f1c-4f6e-9be0-1b3c17e52847/kube-rbac-proxy-web/0.log" Jul 07 22:06:37.989688 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:37.989657 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3321abe-0f1c-4f6e-9be0-1b3c17e52847/kube-rbac-proxy/0.log" Jul 07 22:06:38.030220 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.030187 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3321abe-0f1c-4f6e-9be0-1b3c17e52847/kube-rbac-proxy-thanos/0.log" Jul 07 22:06:38.058067 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.058041 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3321abe-0f1c-4f6e-9be0-1b3c17e52847/init-config-reloader/0.log" Jul 07 22:06:38.097424 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.097389 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-tnw2d_67fc7ebc-6c92-41b7-9122-58b183997f93/prometheus-operator/0.log" Jul 07 22:06:38.130679 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.130632 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-tnw2d_67fc7ebc-6c92-41b7-9122-58b183997f93/kube-rbac-proxy/0.log" Jul 07 22:06:38.169479 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.169442 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-ckzpq_a27db392-9562-49ea-a86c-325f3635da98/prometheus-operator-admission-webhook/0.log" Jul 07 22:06:38.211138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.210971 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7fdf768446-ttr6s_ffd89220-8f5a-4719-8853-7399c5d299e4/telemeter-client/0.log" Jul 07 22:06:38.244034 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.244014 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7fdf768446-ttr6s_ffd89220-8f5a-4719-8853-7399c5d299e4/reload/0.log" Jul 07 22:06:38.277331 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.277286 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7fdf768446-ttr6s_ffd89220-8f5a-4719-8853-7399c5d299e4/kube-rbac-proxy/0.log" Jul 07 22:06:38.317945 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.317915 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc86b697-n572z_d746d876-e77b-493c-bcf8-3fe98d15d59f/thanos-query/0.log" Jul 07 22:06:38.348084 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.348000 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc86b697-n572z_d746d876-e77b-493c-bcf8-3fe98d15d59f/kube-rbac-proxy-web/0.log" Jul 07 22:06:38.374632 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.374589 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc86b697-n572z_d746d876-e77b-493c-bcf8-3fe98d15d59f/kube-rbac-proxy/0.log" Jul 07 22:06:38.405328 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.405298 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc86b697-n572z_d746d876-e77b-493c-bcf8-3fe98d15d59f/prom-label-proxy/0.log" Jul 07 22:06:38.445640 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.445606 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc86b697-n572z_d746d876-e77b-493c-bcf8-3fe98d15d59f/kube-rbac-proxy-rules/0.log" Jul 07 22:06:38.484986 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:38.484953 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc86b697-n572z_d746d876-e77b-493c-bcf8-3fe98d15d59f/kube-rbac-proxy-metrics/0.log" Jul 07 22:06:39.881060 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:39.881026 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx"] Jul 07 22:06:39.887131 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:39.887106 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:39.894302 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:39.894275 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx"] Jul 07 22:06:40.006401 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.006361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-lib-modules\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.006622 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.006417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-sys\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.006622 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.006468 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6gvw\" (UniqueName: \"kubernetes.io/projected/dbf69f23-ffdc-4430-958b-936c9bd4a78e-kube-api-access-l6gvw\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.006622 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.006497 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-proc\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.006622 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.006561 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-podres\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.107481 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.107438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-lib-modules\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.107690 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.107553 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-sys\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.107690 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.107613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6gvw\" (UniqueName: \"kubernetes.io/projected/dbf69f23-ffdc-4430-958b-936c9bd4a78e-kube-api-access-l6gvw\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.107690 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.107655 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-proc\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.107690 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.107667 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-lib-modules\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.107899 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.107701 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-podres\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.107899 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.107678 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-sys\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.107899 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.107874 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-proc\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.108045 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.107978 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbf69f23-ffdc-4430-958b-936c9bd4a78e-podres\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.116596 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.116569 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6gvw\" (UniqueName: \"kubernetes.io/projected/dbf69f23-ffdc-4430-958b-936c9bd4a78e-kube-api-access-l6gvw\") pod \"perf-node-gather-daemonset-nfwcx\" (UID: \"dbf69f23-ffdc-4430-958b-936c9bd4a78e\") " pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.199697 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.199609 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:40.356763 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:40.356734 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx"] Jul 07 22:06:40.360608 ip-10-0-129-68 kubenswrapper[2580]: W0707 22:06:40.360539 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddbf69f23_ffdc_4430_958b_936c9bd4a78e.slice/crio-ca7bd4c3f70cd55141ae9e986931ebe4fb7cbb38d8e343d10c7f7be099a5fd1a WatchSource:0}: Error finding container ca7bd4c3f70cd55141ae9e986931ebe4fb7cbb38d8e343d10c7f7be099a5fd1a: Status 404 returned error can't find the container with id ca7bd4c3f70cd55141ae9e986931ebe4fb7cbb38d8e343d10c7f7be099a5fd1a Jul 07 22:06:41.345574 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:41.345541 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" event={"ID":"dbf69f23-ffdc-4430-958b-936c9bd4a78e","Type":"ContainerStarted","Data":"94250b9aa7863aa82942fe95d1daa8f7132a411a1676a2e37c0cadd79639bb8c"} Jul 07 22:06:41.345958 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:41.345590 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" event={"ID":"dbf69f23-ffdc-4430-958b-936c9bd4a78e","Type":"ContainerStarted","Data":"ca7bd4c3f70cd55141ae9e986931ebe4fb7cbb38d8e343d10c7f7be099a5fd1a"} Jul 07 22:06:41.345958 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:41.345637 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:41.364736 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:41.364689 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" podStartSLOduration=2.364676769 podStartE2EDuration="2.364676769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:06:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:06:41.362818772 +0000 UTC m=+270.926917192" watchObservedRunningTime="2026-07-07 22:06:41.364676769 +0000 UTC m=+270.928775195" Jul 07 22:06:41.604625 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:41.604556 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-km8w9_f8b95043-d806-49a7-baeb-451b8d7fac47/dns/0.log" Jul 07 22:06:41.626838 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:41.626816 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-km8w9_f8b95043-d806-49a7-baeb-451b8d7fac47/kube-rbac-proxy/0.log" Jul 07 22:06:41.723116 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:41.723088 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hfthf_eaf8d762-016e-43f4-bcce-ffd737a1641a/dns-node-resolver/0.log" Jul 07 22:06:42.164094 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:42.164064 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7sxcg_c41ea711-d80d-4533-a02f-5328426cedb9/node-ca/0.log" Jul 07 22:06:43.224502 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:43.224473 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5zv97_0fdba9d1-084e-419a-badb-52a97c802694/serve-healthcheck-canary/0.log" Jul 07 22:06:43.811307 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:43.811283 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vjl8c_96de9fd5-aa10-4eca-ac08-b8e51cac60b1/kube-rbac-proxy/0.log" Jul 07 22:06:43.833576 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:43.833545 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vjl8c_96de9fd5-aa10-4eca-ac08-b8e51cac60b1/exporter/0.log" Jul 07 22:06:43.856746 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:43.856729 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vjl8c_96de9fd5-aa10-4eca-ac08-b8e51cac60b1/extractor/0.log" Jul 07 22:06:47.361678 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:47.361649 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sbz4h/perf-node-gather-daemonset-nfwcx" Jul 07 22:06:49.525637 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:49.525608 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zd4nf_7285ba25-8291-4c0a-acd4-b986c4652f41/kube-multus-additional-cni-plugins/0.log" Jul 07 22:06:49.549003 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:49.548974 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zd4nf_7285ba25-8291-4c0a-acd4-b986c4652f41/egress-router-binary-copy/0.log" Jul 07 22:06:49.573138 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:49.573117 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zd4nf_7285ba25-8291-4c0a-acd4-b986c4652f41/cni-plugins/0.log" Jul 07 22:06:49.598881 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:49.598861 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zd4nf_7285ba25-8291-4c0a-acd4-b986c4652f41/bond-cni-plugin/0.log" Jul 07 22:06:49.622585 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:49.622565 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zd4nf_7285ba25-8291-4c0a-acd4-b986c4652f41/routeoverride-cni/0.log" Jul 07 22:06:49.646533 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:49.646499 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zd4nf_7285ba25-8291-4c0a-acd4-b986c4652f41/whereabouts-cni-bincopy/0.log" Jul 07 22:06:49.674380 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:49.674317 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zd4nf_7285ba25-8291-4c0a-acd4-b986c4652f41/whereabouts-cni/0.log" Jul 07 22:06:49.764416 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:49.764387 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wq9vl_ae3c8757-074f-43f8-9f32-5a69e78843b9/kube-multus/0.log" Jul 07 22:06:49.889175 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:49.889147 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k9ljx_af520e56-7ba8-4259-a907-7a5a28c41e5b/network-metrics-daemon/0.log" Jul 07 22:06:49.913419 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:49.913384 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k9ljx_af520e56-7ba8-4259-a907-7a5a28c41e5b/kube-rbac-proxy/0.log" Jul 07 22:06:50.671537 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:50.671495 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/ovn-controller/0.log" Jul 07 22:06:50.696069 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:50.696043 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/ovn-acl-logging/0.log" Jul 07 22:06:50.696069 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:50.696059 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/ovn-acl-logging/1.log" Jul 07 22:06:50.716826 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:50.716797 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/kube-rbac-proxy-node/0.log" Jul 07 22:06:50.741391 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:50.741304 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/kube-rbac-proxy-ovn-metrics/0.log" Jul 07 22:06:50.767314 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:50.767283 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/northd/0.log" Jul 07 22:06:50.793785 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:50.793759 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/nbdb/0.log" Jul 07 22:06:50.817027 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:50.817001 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/sbdb/0.log" Jul 07 22:06:50.862641 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:50.862612 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hk92b_bc4474f8-d5cd-49ee-9618-cbf22bdbcbbc/ovnkube-controller/0.log" Jul 07 22:06:52.308498 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:52.308471 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-qsxfk_9b236333-c490-4f99-b91f-d44b01461c72/network-check-target-container/0.log" Jul 07 22:06:53.187305 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:53.187274 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-glszg_0fc51f9e-2d89-4649-8992-87f23e1e7721/iptables-alerter/0.log" Jul 07 22:06:53.829390 ip-10-0-129-68 kubenswrapper[2580]: I0707 22:06:53.829330 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-b4fxx_43af0cd5-5dfe-45f2-a50e-84bcda42d705/tuned/0.log"