Jul 15 21:27:35.008668 ip-10-0-130-62 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 21:27:35.008678 ip-10-0-130-62 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 21:27:35.008690 ip-10-0-130-62 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 21:27:35.009022 ip-10-0-130-62 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 21:27:45.257864 ip-10-0-130-62 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 15 21:27:45.257882 ip-10-0-130-62 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e4dba96d96674a8d9b8e041c3f208253 -- Jul 15 21:29:18.364380 ip-10-0-130-62 systemd[1]: Starting Kubernetes Kubelet... Jul 15 21:29:18.824209 ip-10-0-130-62 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 21:29:18.824209 ip-10-0-130-62 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 21:29:18.824209 ip-10-0-130-62 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 21:29:18.824209 ip-10-0-130-62 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 21:29:18.824209 ip-10-0-130-62 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 21:29:18.824209 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.818284 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827775 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827792 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827796 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827799 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827802 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827805 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827809 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827812 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827815 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827818 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827822 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827824 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827827 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827830 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827832 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827835 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827837 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827840 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827842 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:29:18.917971 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827845 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:29:18.902518 ip-10-0-130-62 systemd[1]: Started Kubernetes Kubelet. Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827847 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827850 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827853 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827863 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827866 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827868 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827871 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827874 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827876 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827879 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827881 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827884 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827887 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827891 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827895 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827899 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827902 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827905 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827908 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:29:19.156334 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827910 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827913 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827915 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827918 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827920 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827924 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827926 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827929 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827931 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827934 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827937 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827939 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827942 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827945 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827947 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827950 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827952 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827955 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827958 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:29:19.167487 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827960 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827963 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827965 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827968 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827970 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827973 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827975 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827978 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827980 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827984 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827989 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827992 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827995 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.827998 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828001 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828004 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828007 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828009 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828011 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:29:19.168811 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828014 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828016 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828018 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828021 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828024 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828026 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828030 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828033 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828035 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828038 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828040 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828042 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828045 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828048 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828052 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828055 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828057 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828060 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828063 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828065 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:29:19.169770 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828068 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.828071 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829794 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829802 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829806 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829809 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829812 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829815 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829818 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829821 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829823 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829826 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829831 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829835 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829838 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829841 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829844 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829847 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829849 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:29:19.170385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829852 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829855 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829858 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829861 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829863 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829866 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829868 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829873 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829875 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829878 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829880 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829883 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829885 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829888 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829890 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829893 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829895 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829898 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829901 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829903 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:29:19.171000 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829906 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829908 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829911 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829913 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829916 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829920 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829923 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829925 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829929 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829931 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829934 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829937 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829939 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829942 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829945 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829947 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829950 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829952 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829954 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829959 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:29:19.171616 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829961 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829964 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829966 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829969 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829971 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829974 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829976 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829979 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829982 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829984 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829987 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829989 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829992 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829995 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829997 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.829999 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830002 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830009 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830012 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:29:19.172249 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830015 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830018 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830021 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830024 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830027 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830029 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830032 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830035 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830039 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830043 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830045 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830048 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830052 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830055 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830058 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830061 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830063 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830066 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830069 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:29:19.172789 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830071 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830074 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830076 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830079 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830499 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830504 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830507 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830509 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830512 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830515 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830517 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830521 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830525 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830529 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830531 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830534 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830536 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830539 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830542 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:29:19.173413 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830544 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830548 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830551 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830554 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830556 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830559 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830562 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830566 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830568 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830571 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830573 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830576 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830578 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830581 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830583 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830585 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830588 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830590 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830595 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:29:19.173973 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830598 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830602 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830604 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830607 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830610 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830614 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830617 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830619 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830622 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830624 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830627 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830629 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830632 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830634 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830636 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830639 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830642 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830645 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830647 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830651 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:29:19.174593 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830653 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830656 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830658 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830661 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830663 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830667 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830670 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830673 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830676 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830678 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830680 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830684 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830687 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830689 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830691 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830694 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830696 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830700 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830702 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830705 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:29:19.175166 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830707 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830709 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830712 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830714 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830716 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830719 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830721 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830724 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830726 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830729 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830731 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830735 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830737 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830740 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830742 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830745 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830747 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830750 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830752 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:29:19.175796 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830755 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830757 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830759 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830762 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830765 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.830767 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830835 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830842 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830849 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830853 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830859 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830862 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830866 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830870 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830873 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830876 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830879 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830883 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830886 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830889 2578 flags.go:64] FLAG: --cgroup-root="" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830891 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830894 2578 flags.go:64] FLAG: --client-ca-file="" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830898 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 15 21:29:19.176343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830901 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830906 2578 flags.go:64] FLAG: --cluster-domain="" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830909 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830912 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830915 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830918 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830922 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830924 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830927 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830930 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830933 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830936 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830939 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830942 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830945 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830949 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830952 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830954 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830957 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830961 2578 flags.go:64] FLAG: --enable-server="true" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830964 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830968 2578 flags.go:64] FLAG: --event-burst="100" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830972 2578 flags.go:64] FLAG: --event-qps="50" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830975 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 21:29:19.176958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830977 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830981 2578 flags.go:64] FLAG: --eviction-hard="" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830984 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830987 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830990 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830993 2578 flags.go:64] FLAG: --eviction-soft="" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830996 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.830999 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831002 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831008 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831011 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831014 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831017 2578 flags.go:64] FLAG: --feature-gates="" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831021 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831024 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831027 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831030 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831033 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831036 2578 flags.go:64] FLAG: --help="false" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831039 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831042 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831045 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831048 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831051 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 21:29:19.177656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831054 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831057 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831060 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831064 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831067 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831070 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831073 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831076 2578 flags.go:64] FLAG: --kube-reserved="" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831078 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831081 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831084 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831101 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831105 2578 flags.go:64] FLAG: --lock-file="" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831108 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831110 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831113 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831118 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831123 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831126 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831129 2578 flags.go:64] FLAG: --logging-format="text" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831132 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831135 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831138 2578 flags.go:64] FLAG: --manifest-url="" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831141 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831145 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 21:29:19.178399 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831148 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831152 2578 flags.go:64] FLAG: --max-pods="110" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831155 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831158 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831161 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831164 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831167 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831170 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831172 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831180 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831184 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831187 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831190 2578 flags.go:64] FLAG: --pod-cidr="" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831193 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831199 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831202 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831205 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831208 2578 flags.go:64] FLAG: --port="10250" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831211 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831214 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0339bb133e854951a" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831217 2578 flags.go:64] FLAG: --qos-reserved="" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831220 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831223 2578 flags.go:64] FLAG: --register-node="true" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831226 2578 flags.go:64] FLAG: --register-with-taints="" Jul 15 21:29:19.179028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831229 2578 flags.go:64] FLAG: --registry-burst="10" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831234 2578 flags.go:64] FLAG: --registry-qps="5" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831236 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831239 2578 flags.go:64] FLAG: --reserved-memory="" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831243 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831246 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831249 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831252 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831255 2578 flags.go:64] FLAG: --runonce="false" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831258 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831261 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831264 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831267 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831269 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831272 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831276 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831279 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831281 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831286 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831289 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831292 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831295 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831298 2578 flags.go:64] FLAG: --system-cgroups="" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831301 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831306 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 21:29:19.179720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831308 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831311 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831315 2578 flags.go:64] FLAG: --tls-min-version="" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831318 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831321 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831324 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831327 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831329 2578 flags.go:64] FLAG: --v="2" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831335 2578 flags.go:64] FLAG: --version="false" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831342 2578 flags.go:64] FLAG: --vmodule="" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831346 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.831349 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831463 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831467 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831469 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831472 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831475 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831478 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831480 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831482 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831485 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831488 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831490 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:29:19.180660 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831492 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831495 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831499 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831501 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831504 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831507 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831509 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831512 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831515 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831517 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831519 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831522 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831524 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831527 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831529 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831531 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831536 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831540 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831543 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:29:19.183557 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831547 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831549 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831552 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831555 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831557 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831560 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831562 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831564 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831567 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831569 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831572 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831575 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831577 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831579 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831582 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831586 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831589 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831592 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831596 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:29:19.392725 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831598 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831601 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831603 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831606 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831608 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831611 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831613 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831616 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831618 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831621 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831623 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831627 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831630 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831632 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831635 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831638 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831640 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831642 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831645 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831647 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:29:19.393370 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831650 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831652 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831655 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831657 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831659 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831662 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831664 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831667 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831670 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831673 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831675 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831678 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831681 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831683 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831686 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831688 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831690 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831693 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831695 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831698 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831701 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:29:19.393925 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831703 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831706 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831710 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831713 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831716 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831719 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831723 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831726 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.831729 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.832519 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.840537 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.840552 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.841556 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.841564 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 21:29:19.394548 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841627 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841632 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841635 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841640 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841644 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841650 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841653 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841656 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841659 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841662 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841665 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841668 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841671 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841674 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841677 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841680 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841683 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841685 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841688 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:29:19.394918 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841690 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841693 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841698 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841700 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841703 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841706 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841709 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841711 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841714 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841716 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841719 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841721 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841724 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841727 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841729 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841731 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841734 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841736 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841741 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841743 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:29:19.395511 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841746 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841748 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841750 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841753 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841756 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841759 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841761 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841763 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841766 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841769 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841771 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841774 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841776 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841780 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841785 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841788 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841790 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841792 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841796 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841799 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:29:19.396027 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841801 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841804 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841806 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841809 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841811 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841814 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841816 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841819 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841821 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841824 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841828 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841831 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841833 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841836 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841838 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841841 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841843 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841846 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841848 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841851 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:29:19.396581 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841853 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841856 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841859 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841861 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841864 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841866 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841870 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841874 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841877 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841880 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841883 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841886 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841889 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841891 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841894 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841896 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841899 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841902 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841904 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:29:19.397159 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.841907 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.841913 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842027 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842033 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842036 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842039 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842042 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842045 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842049 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842051 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842054 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842057 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842060 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842063 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842066 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:29:19.397689 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842068 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842071 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842073 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842075 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842078 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842083 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842098 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842101 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842104 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842106 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842110 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842112 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842115 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842118 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842120 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842123 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842126 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842128 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842131 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842134 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:29:19.398123 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842138 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842141 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842143 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842146 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842149 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842151 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842154 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842157 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842159 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842162 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842165 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842167 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842171 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842175 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842178 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842181 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842183 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842187 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842190 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:29:19.398694 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842192 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842195 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842197 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842200 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842203 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842206 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842208 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842211 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842213 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842216 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842219 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842221 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842225 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842229 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842232 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842236 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842238 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842241 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842244 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:29:19.399237 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842246 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842248 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842251 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842254 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842256 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842258 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842261 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842264 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842266 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842269 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842271 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842275 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842278 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842280 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842283 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842298 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842301 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842303 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842307 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842309 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:29:19.399765 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842313 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842316 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842321 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842323 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842326 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842328 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842332 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:18.842335 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.842340 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.842458 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.844970 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.846771 2578 server.go:1019] "Starting client certificate rotation" Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.846868 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 21:29:19.400296 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.846908 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 21:29:19.400675 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.849537 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 21:29:19.400675 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.852641 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 21:29:19.400675 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.865339 2578 log.go:25] "Validated CRI v1 runtime API" Jul 15 21:29:19.400675 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.867810 2578 log.go:25] "Validated CRI v1 image API" Jul 15 21:29:19.400675 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.872295 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 21:29:19.400675 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.874722 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 21:29:19.400675 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.874779 2578 fs.go:135] Filesystem UUIDs: map[3c870de8-4f23-4416-ab44-90c1d6f44b4c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a6740c5c-bac9-4885-8308-1fd49a5fb37d:/dev/nvme0n1p3] Jul 15 21:29:19.400675 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.874794 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.879890 2578 manager.go:216] Machine: {Timestamp:2026-07-15 21:29:18.878463282 +0000 UTC m=+0.381898567 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3058537 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28e539e25e196d6d379522c149f39c SystemUUID:ec28e539-e25e-196d-6d37-9522c149f39c BootID:e4dba96d-9667-4a8d-9b8e-041c3f208253 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:01:dd:13:c5:01 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:01:dd:13:c5:01 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:a2:69:14:11:81 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.879992 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.880071 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.883549 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.883571 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-62.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.884422 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.884432 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.884439 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.884453 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.886300 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.890997 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.891118 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.891146 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.891155 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.891173 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.891181 2578 kubelet.go:398] "Adding apiserver pod source" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.891192 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 21:29:19.400991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.891528 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.891541 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.893920 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.896426 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.896460 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.896607 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.896619 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.896625 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.896630 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.896638 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.896644 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.897262 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.897268 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.897275 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.897280 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.897293 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.897301 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.898146 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.898153 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.900336 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7blj" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.901180 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-62.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:18.901433 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-62.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:18.901439 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.901758 2578 server.go:1268] "Started kubelet" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.902251 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.904450 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.905364 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.905403 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.908834 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.909567 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7blj" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:18.914539 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:18.913648 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-62.ec2.internal.18c293d82302a855 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-62.ec2.internal,UID:ip-10-0-130-62.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-62.ec2.internal,},FirstTimestamp:2026-07-15 21:29:18.901733461 +0000 UTC m=+0.405168752,LastTimestamp:2026-07-15 21:29:18.901733461 +0000 UTC m=+0.405168752,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-62.ec2.internal,}" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.916992 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.917020 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:18.917137 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.917862 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.918129 2578 factory.go:153] Registering CRI-O factory Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.918143 2578 factory.go:223] Registration of the crio container factory successfully Jul 15 21:29:19.401565 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.918191 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.918200 2578 factory.go:55] Registering systemd factory Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.918208 2578 factory.go:223] Registration of the systemd container factory successfully Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.918229 2578 factory.go:103] Registering Raw factory Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.918242 2578 manager.go:1196] Started watching for new ooms in manager Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.918661 2578 manager.go:349] Starting recovery of all containers Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:18.918796 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-62.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.918869 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.918885 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.919684 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.919692 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:18.918809 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.927447 2578 manager.go:354] Recovery completed Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.932280 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.934254 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.934284 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.934300 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.935831 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.935840 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.935856 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.936420 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.937573 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.937601 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.937614 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.938066 2578 policy_none.go:49] "None policy: Start" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.938081 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.938157 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:18.940226 2578 policy_none.go:47] "Start" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.005799 2578 manager.go:337] "Starting Device Plugin manager" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.005833 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.005844 2578 server.go:87] "Starting device plugin registration server" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.006158 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.006171 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.006281 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.006353 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.006360 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.006831 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.006872 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.043225 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.044431 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.044480 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.044507 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.044546 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.046512 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.106560 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.107598 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.107629 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.107639 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.107664 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.402617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.113308 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.113333 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-62.ec2.internal\": node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.128244 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.145616 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-62.ec2.internal"] Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.145693 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.146701 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.146724 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.146733 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.149121 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.149202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.149239 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.149855 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.149881 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.149892 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.150394 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.150427 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.150440 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.152555 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.152575 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.157145 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.157169 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.157194 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.162683 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-62.ec2.internal\" not found" node="ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.168487 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-62.ec2.internal\" not found" node="ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.221884 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cc89e18af835f63bc098b914e5f0f33d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal\" (UID: \"cc89e18af835f63bc098b914e5f0f33d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.221911 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc89e18af835f63bc098b914e5f0f33d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal\" (UID: \"cc89e18af835f63bc098b914e5f0f33d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.221927 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40d5eee52fbcaf2e5bbfd3cc3c45fbae-config\") pod \"kube-apiserver-proxy-ip-10-0-130-62.ec2.internal\" (UID: \"40d5eee52fbcaf2e5bbfd3cc3c45fbae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.228984 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.322320 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cc89e18af835f63bc098b914e5f0f33d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal\" (UID: \"cc89e18af835f63bc098b914e5f0f33d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.552949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.322356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc89e18af835f63bc098b914e5f0f33d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal\" (UID: \"cc89e18af835f63bc098b914e5f0f33d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.554726 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.322379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40d5eee52fbcaf2e5bbfd3cc3c45fbae-config\") pod \"kube-apiserver-proxy-ip-10-0-130-62.ec2.internal\" (UID: \"40d5eee52fbcaf2e5bbfd3cc3c45fbae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.554726 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.322425 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40d5eee52fbcaf2e5bbfd3cc3c45fbae-config\") pod \"kube-apiserver-proxy-ip-10-0-130-62.ec2.internal\" (UID: \"40d5eee52fbcaf2e5bbfd3cc3c45fbae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.554726 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.322430 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc89e18af835f63bc098b914e5f0f33d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal\" (UID: \"cc89e18af835f63bc098b914e5f0f33d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.554726 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.322425 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cc89e18af835f63bc098b914e5f0f33d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal\" (UID: \"cc89e18af835f63bc098b914e5f0f33d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.554726 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.329422 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:19.554726 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.429792 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:19.554726 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.464951 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.554726 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.471809 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-62.ec2.internal" Jul 15 21:29:19.554726 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.529892 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:19.751412 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.630933 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:19.751412 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.731374 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:19.967582 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.831708 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:19.967582 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.845895 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 21:29:19.967582 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.846093 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 21:29:19.967582 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.913241 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 21:29:19.967582 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.915286 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 21:24:19 +0000 UTC" deadline="2028-04-30 17:20:06.010041956 +0000 UTC" Jul 15 21:29:19.967582 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:19.915313 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15715h50m46.094732183s" Jul 15 21:29:19.967582 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:19.931937 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:20.157604 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:20.031998 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:20.157604 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:20.133139 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:20.321039 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:20.234058 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-62.ec2.internal\" not found" Jul 15 21:29:20.321039 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.276895 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 21:29:20.321039 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:20.303771 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc89e18af835f63bc098b914e5f0f33d.slice/crio-2fcb9fc8752fd6ed8831437bbfa971021fd819779347cb9ac707d9f1975ef348 WatchSource:0}: Error finding container 2fcb9fc8752fd6ed8831437bbfa971021fd819779347cb9ac707d9f1975ef348: Status 404 returned error can't find the container with id 2fcb9fc8752fd6ed8831437bbfa971021fd819779347cb9ac707d9f1975ef348 Jul 15 21:29:20.326245 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.323733 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" Jul 15 21:29:20.326245 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:20.326161 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40d5eee52fbcaf2e5bbfd3cc3c45fbae.slice/crio-1f077884edf7e7fb0d0b41f6f7190da221c8b04c31e2a809b4e1f713dbdda645 WatchSource:0}: Error finding container 1f077884edf7e7fb0d0b41f6f7190da221c8b04c31e2a809b4e1f713dbdda645: Status 404 returned error can't find the container with id 1f077884edf7e7fb0d0b41f6f7190da221c8b04c31e2a809b4e1f713dbdda645 Jul 15 21:29:20.326612 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.326302 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:29:20.487843 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.329896 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 21:29:20.487843 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.334327 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 21:29:20.487843 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.335287 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-62.ec2.internal" Jul 15 21:29:20.487843 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.343598 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 21:29:20.892319 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.892015 2578 apiserver.go:52] "Watching apiserver" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.916668 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.917089 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-rfpjx","openshift-multus/network-metrics-daemon-j7s97","openshift-network-operator/iptables-alerter-fk4ld","kube-system/konnectivity-agent-lbd68","kube-system/kube-apiserver-proxy-ip-10-0-130-62.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj","openshift-image-registry/node-ca-822j5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal","openshift-multus/multus-lbm84","openshift-network-diagnostics/network-check-target-t9g8r","openshift-ovn-kubernetes/ovnkube-node-hqhrr","kube-system/global-pull-secret-syncer-6jqqs","openshift-cluster-node-tuning-operator/tuned-bl2b4","openshift-dns/node-resolver-nbzd8"] Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.918892 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.919878 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-822j5" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.922658 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.922747 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.922856 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.923007 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lhxlv\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.923099 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.924902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fk4ld" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.927515 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.928437 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.928496 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.928669 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.928714 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.928443 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.928897 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.929980 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jlxr2\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.930237 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.930475 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.930999 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.931271 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.931456 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-m5jtd\"" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.936108 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f191e06a-8191-4762-8e60-52d29e065cb2-serviceca\") pod \"node-ca-822j5\" (UID: \"f191e06a-8191-4762-8e60-52d29e065cb2\") " pod="openshift-image-registry/node-ca-822j5" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.936154 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m6dv\" (UniqueName: \"kubernetes.io/projected/f191e06a-8191-4762-8e60-52d29e065cb2-kube-api-access-8m6dv\") pod \"node-ca-822j5\" (UID: \"f191e06a-8191-4762-8e60-52d29e065cb2\") " pod="openshift-image-registry/node-ca-822j5" Jul 15 21:29:20.968029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.936205 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f191e06a-8191-4762-8e60-52d29e065cb2-host\") pod \"node-ca-822j5\" (UID: \"f191e06a-8191-4762-8e60-52d29e065cb2\") " pod="openshift-image-registry/node-ca-822j5" Jul 15 21:29:20.974722 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.973830 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5jt85\"" Jul 15 21:29:20.975538 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.974785 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 21:29:20.975538 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.974947 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nbzd8" Jul 15 21:29:20.975538 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.975384 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.977497 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-96tz9\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.977602 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.977851 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.977914 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gfqdz" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.977932 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gfqdz" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.978046 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.980488 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.980884 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.980946 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p2t5g\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.981094 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.981291 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.984104 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.984396 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.984460 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.984656 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.984703 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5zldt\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.984867 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.986360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:20.986465 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.986812 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.987207 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.987303 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-svqdc\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.988938 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:20.989144 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.991592 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.994763 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:20.994839 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.994887 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.995052 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dmhfn\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:20.995171 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.019010 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 21:29:21.043441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-systemd-units\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037398 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-etc-openvswitch\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037428 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-run-k8s-cni-cncf-io\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dfb01554-4420-4174-a15a-1b67f0dfd799-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037486 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-kubernetes\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037574 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a864031f-a26e-40bb-b130-6d0fe02a72dc-tmp\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037596 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-var-lib-openvswitch\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-cni-bin\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037672 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/09c29bbd-d242-45cc-b652-d1111a3f417e-ovn-node-metrics-cert\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp5jk\" (UniqueName: \"kubernetes.io/projected/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-kube-api-access-hp5jk\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037728 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-lib-modules\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037772 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7zmr\" (UniqueName: \"kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr\") pod \"network-check-target-t9g8r\" (UID: \"0037cbd3-6cda-4518-a2b0-db580fa02577\") " pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037798 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-system-cni-dir\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037817 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/09c29bbd-d242-45cc-b652-d1111a3f417e-env-overrides\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d733c3ef-eaf2-44b5-8e84-c5f17c26665a-konnectivity-ca\") pod \"konnectivity-agent-lbd68\" (UID: \"d733c3ef-eaf2-44b5-8e84-c5f17c26665a\") " pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037865 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d05dda3-b68c-4486-9369-a0615903e606-tmp-dir\") pod \"node-resolver-nbzd8\" (UID: \"4d05dda3-b68c-4486-9369-a0615903e606\") " pod="openshift-dns/node-resolver-nbzd8" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037906 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-hostroot\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-run-multus-certs\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.037977 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-socket-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038000 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-sysconfig\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038025 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d05dda3-b68c-4486-9369-a0615903e606-hosts-file\") pod \"node-resolver-nbzd8\" (UID: \"4d05dda3-b68c-4486-9369-a0615903e606\") " pod="openshift-dns/node-resolver-nbzd8" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-cni-binary-copy\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-etc-kubernetes\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038144 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-cnibin\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038168 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrrk4\" (UniqueName: \"kubernetes.io/projected/dfb01554-4420-4174-a15a-1b67f0dfd799-kube-api-access-vrrk4\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038193 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-run-openvswitch\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038216 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-os-release\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038244 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-host\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.046070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038284 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52934511-13c5-4254-bdbd-b3e67a77c80f-dbus\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038308 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038326 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82e2f4b7-3511-4f83-bb59-06416fd961ae-host-slash\") pod \"iptables-alerter-fk4ld\" (UID: \"82e2f4b7-3511-4f83-bb59-06416fd961ae\") " pod="openshift-network-operator/iptables-alerter-fk4ld" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038363 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmhjh\" (UniqueName: \"kubernetes.io/projected/82e2f4b7-3511-4f83-bb59-06416fd961ae-kube-api-access-fmhjh\") pod \"iptables-alerter-fk4ld\" (UID: \"82e2f4b7-3511-4f83-bb59-06416fd961ae\") " pod="openshift-network-operator/iptables-alerter-fk4ld" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038379 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038409 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/09c29bbd-d242-45cc-b652-d1111a3f417e-ovnkube-config\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038427 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/09c29bbd-d242-45cc-b652-d1111a3f417e-ovnkube-script-lib\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-var-lib-kubelet\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038499 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dfb01554-4420-4174-a15a-1b67f0dfd799-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-modprobe-d\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-tuned\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038559 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dfb01554-4420-4174-a15a-1b67f0dfd799-cni-binary-copy\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038589 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f191e06a-8191-4762-8e60-52d29e065cb2-serviceca\") pod \"node-ca-822j5\" (UID: \"f191e06a-8191-4762-8e60-52d29e065cb2\") " pod="openshift-image-registry/node-ca-822j5" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038618 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-socket-dir-parent\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038643 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-daemon-config\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038682 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-system-cni-dir\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.047292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038707 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-registration-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-sys\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-slash\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038790 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-node-log\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-run-ovn-kubernetes\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs8b4\" (UniqueName: \"kubernetes.io/projected/09c29bbd-d242-45cc-b652-d1111a3f417e-kube-api-access-xs8b4\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038879 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/82e2f4b7-3511-4f83-bb59-06416fd961ae-iptables-alerter-script\") pod \"iptables-alerter-fk4ld\" (UID: \"82e2f4b7-3511-4f83-bb59-06416fd961ae\") " pod="openshift-network-operator/iptables-alerter-fk4ld" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-cni-dir\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038926 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-os-release\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038949 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-conf-dir\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.038986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f191e06a-8191-4762-8e60-52d29e065cb2-host\") pod \"node-ca-822j5\" (UID: \"f191e06a-8191-4762-8e60-52d29e065cb2\") " pod="openshift-image-registry/node-ca-822j5" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039011 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-kubelet\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-run-systemd\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-var-lib-kubelet\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039098 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-sysctl-d\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039123 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-run\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5czx5\" (UniqueName: \"kubernetes.io/projected/50eee8aa-8304-424c-8525-91b270d118ac-kube-api-access-5czx5\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:21.048332 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039185 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-run-ovn\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039210 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-log-socket\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039236 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-run-netns\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039256 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f191e06a-8191-4762-8e60-52d29e065cb2-serviceca\") pod \"node-ca-822j5\" (UID: \"f191e06a-8191-4762-8e60-52d29e065cb2\") " pod="openshift-image-registry/node-ca-822j5" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039262 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkdd9\" (UniqueName: \"kubernetes.io/projected/319df9cd-0ae2-4abf-8da6-ca519d42834e-kube-api-access-fkdd9\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039338 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-systemd\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039412 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-run-netns\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039416 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f191e06a-8191-4762-8e60-52d29e065cb2-host\") pod \"node-ca-822j5\" (UID: \"f191e06a-8191-4762-8e60-52d29e065cb2\") " pod="openshift-image-registry/node-ca-822j5" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039468 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-cnibin\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039496 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-var-lib-cni-bin\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039529 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-sysctl-conf\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m6dv\" (UniqueName: \"kubernetes.io/projected/f191e06a-8191-4762-8e60-52d29e065cb2-kube-api-access-8m6dv\") pod \"node-ca-822j5\" (UID: \"f191e06a-8191-4762-8e60-52d29e065cb2\") " pod="openshift-image-registry/node-ca-822j5" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039591 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-cni-netd\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039616 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtfq8\" (UniqueName: \"kubernetes.io/projected/4d05dda3-b68c-4486-9369-a0615903e606-kube-api-access-mtfq8\") pod \"node-resolver-nbzd8\" (UID: \"4d05dda3-b68c-4486-9369-a0615903e606\") " pod="openshift-dns/node-resolver-nbzd8" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039643 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k55jc\" (UniqueName: \"kubernetes.io/projected/a864031f-a26e-40bb-b130-6d0fe02a72dc-kube-api-access-k55jc\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52934511-13c5-4254-bdbd-b3e67a77c80f-kubelet-config\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039686 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d733c3ef-eaf2-44b5-8e84-c5f17c26665a-agent-certs\") pod \"konnectivity-agent-lbd68\" (UID: \"d733c3ef-eaf2-44b5-8e84-c5f17c26665a\") " pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:21.049677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-var-lib-cni-multus\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.050707 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-device-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.050707 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.039760 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-sys-fs\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.050707 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.045389 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 21:29:21.050707 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.049843 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 21:29:21.053612 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.051863 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-62.ec2.internal" event={"ID":"40d5eee52fbcaf2e5bbfd3cc3c45fbae","Type":"ContainerStarted","Data":"1f077884edf7e7fb0d0b41f6f7190da221c8b04c31e2a809b4e1f713dbdda645"} Jul 15 21:29:21.053612 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.053553 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" event={"ID":"cc89e18af835f63bc098b914e5f0f33d","Type":"ContainerStarted","Data":"2fcb9fc8752fd6ed8831437bbfa971021fd819779347cb9ac707d9f1975ef348"} Jul 15 21:29:21.141400 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.084764 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m6dv\" (UniqueName: \"kubernetes.io/projected/f191e06a-8191-4762-8e60-52d29e065cb2-kube-api-access-8m6dv\") pod \"node-ca-822j5\" (UID: \"f191e06a-8191-4762-8e60-52d29e065cb2\") " pod="openshift-image-registry/node-ca-822j5" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.144817 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-cni-netd\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.144864 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtfq8\" (UniqueName: \"kubernetes.io/projected/4d05dda3-b68c-4486-9369-a0615903e606-kube-api-access-mtfq8\") pod \"node-resolver-nbzd8\" (UID: \"4d05dda3-b68c-4486-9369-a0615903e606\") " pod="openshift-dns/node-resolver-nbzd8" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.144896 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k55jc\" (UniqueName: \"kubernetes.io/projected/a864031f-a26e-40bb-b130-6d0fe02a72dc-kube-api-access-k55jc\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.144921 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52934511-13c5-4254-bdbd-b3e67a77c80f-kubelet-config\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.144951 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d733c3ef-eaf2-44b5-8e84-c5f17c26665a-agent-certs\") pod \"konnectivity-agent-lbd68\" (UID: \"d733c3ef-eaf2-44b5-8e84-c5f17c26665a\") " pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.144954 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-cni-netd\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.144977 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-var-lib-cni-multus\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-device-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145036 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-var-lib-cni-multus\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145053 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-sys-fs\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145098 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52934511-13c5-4254-bdbd-b3e67a77c80f-kubelet-config\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145108 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-systemd-units\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145115 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-device-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145143 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-etc-openvswitch\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145164 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-systemd-units\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145189 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-run-k8s-cni-cncf-io\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.145383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145221 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-sys-fs\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dfb01554-4420-4174-a15a-1b67f0dfd799-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145256 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-kubernetes\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145310 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a864031f-a26e-40bb-b130-6d0fe02a72dc-tmp\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145321 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-run-k8s-cni-cncf-io\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145376 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-etc-openvswitch\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145447 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-var-lib-openvswitch\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-cni-bin\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145592 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/09c29bbd-d242-45cc-b652-d1111a3f417e-ovn-node-metrics-cert\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp5jk\" (UniqueName: \"kubernetes.io/projected/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-kube-api-access-hp5jk\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145635 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-var-lib-openvswitch\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145683 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-lib-modules\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7zmr\" (UniqueName: \"kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr\") pod \"network-check-target-t9g8r\" (UID: \"0037cbd3-6cda-4518-a2b0-db580fa02577\") " pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:21.149038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145762 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-lib-modules\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145766 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-kubernetes\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145763 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-system-cni-dir\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/09c29bbd-d242-45cc-b652-d1111a3f417e-env-overrides\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145818 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-system-cni-dir\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145829 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d733c3ef-eaf2-44b5-8e84-c5f17c26665a-konnectivity-ca\") pod \"konnectivity-agent-lbd68\" (UID: \"d733c3ef-eaf2-44b5-8e84-c5f17c26665a\") " pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d05dda3-b68c-4486-9369-a0615903e606-tmp-dir\") pod \"node-resolver-nbzd8\" (UID: \"4d05dda3-b68c-4486-9369-a0615903e606\") " pod="openshift-dns/node-resolver-nbzd8" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-hostroot\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145905 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-run-multus-certs\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145930 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-socket-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-sysconfig\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.145982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.146017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d05dda3-b68c-4486-9369-a0615903e606-hosts-file\") pod \"node-resolver-nbzd8\" (UID: \"4d05dda3-b68c-4486-9369-a0615903e606\") " pod="openshift-dns/node-resolver-nbzd8" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.146052 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-cni-binary-copy\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.146141 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-run-multus-certs\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.146226 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dfb01554-4420-4174-a15a-1b67f0dfd799-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.146285 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-cni-bin\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.146367 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:21.150130 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.146709 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/09c29bbd-d242-45cc-b652-d1111a3f417e-env-overrides\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.154376 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.147097 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d733c3ef-eaf2-44b5-8e84-c5f17c26665a-konnectivity-ca\") pod \"konnectivity-agent-lbd68\" (UID: \"d733c3ef-eaf2-44b5-8e84-c5f17c26665a\") " pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.155621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-etc-kubernetes\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.155673 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret podName:52934511-13c5-4254-bdbd-b3e67a77c80f nodeName:}" failed. No retries permitted until 2026-07-15 21:29:21.646417043 +0000 UTC m=+3.149852332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret") pod "global-pull-secret-syncer-6jqqs" (UID: "52934511-13c5-4254-bdbd-b3e67a77c80f") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.155751 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-cnibin\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.155808 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrrk4\" (UniqueName: \"kubernetes.io/projected/dfb01554-4420-4174-a15a-1b67f0dfd799-kube-api-access-vrrk4\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.155845 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-run-openvswitch\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.155916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-os-release\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.155970 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.156005 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-host\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.156020 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-socket-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.156121 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-cnibin\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.156204 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-os-release\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.156340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.156287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-sysconfig\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.157086 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.155500 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d05dda3-b68c-4486-9369-a0615903e606-tmp-dir\") pod \"node-resolver-nbzd8\" (UID: \"4d05dda3-b68c-4486-9369-a0615903e606\") " pod="openshift-dns/node-resolver-nbzd8" Jul 15 21:29:21.157086 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.156479 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.157086 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.156500 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-run-openvswitch\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.157086 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.156516 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-hostroot\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.157086 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.156560 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-host\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.157086 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.156675 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d05dda3-b68c-4486-9369-a0615903e606-hosts-file\") pod \"node-resolver-nbzd8\" (UID: \"4d05dda3-b68c-4486-9369-a0615903e606\") " pod="openshift-dns/node-resolver-nbzd8" Jul 15 21:29:21.157086 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.156715 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a864031f-a26e-40bb-b130-6d0fe02a72dc-tmp\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.157086 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.156821 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:29:21.157086 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.156972 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:29:21.157086 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.157016 2578 projected.go:196] Error preparing data for projected volume kube-api-access-d7zmr for pod openshift-network-diagnostics/network-check-target-t9g8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.157142 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr podName:0037cbd3-6cda-4518-a2b0-db580fa02577 nodeName:}" failed. No retries permitted until 2026-07-15 21:29:21.657115231 +0000 UTC m=+3.160550527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-d7zmr" (UniqueName: "kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr") pod "network-check-target-t9g8r" (UID: "0037cbd3-6cda-4518-a2b0-db580fa02577") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.157193 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52934511-13c5-4254-bdbd-b3e67a77c80f-dbus\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.158266 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-cni-binary-copy\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.158674 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d733c3ef-eaf2-44b5-8e84-c5f17c26665a-agent-certs\") pod \"konnectivity-agent-lbd68\" (UID: \"d733c3ef-eaf2-44b5-8e84-c5f17c26665a\") " pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.158832 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-etc-kubernetes\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.160525 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52934511-13c5-4254-bdbd-b3e67a77c80f-dbus\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.161437 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.163566 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/09c29bbd-d242-45cc-b652-d1111a3f417e-ovn-node-metrics-cert\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.164297 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k55jc\" (UniqueName: \"kubernetes.io/projected/a864031f-a26e-40bb-b130-6d0fe02a72dc-kube-api-access-k55jc\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.164582 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtfq8\" (UniqueName: \"kubernetes.io/projected/4d05dda3-b68c-4486-9369-a0615903e606-kube-api-access-mtfq8\") pod \"node-resolver-nbzd8\" (UID: \"4d05dda3-b68c-4486-9369-a0615903e606\") " pod="openshift-dns/node-resolver-nbzd8" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.167049 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp5jk\" (UniqueName: \"kubernetes.io/projected/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-kube-api-access-hp5jk\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.169135 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrrk4\" (UniqueName: \"kubernetes.io/projected/dfb01554-4420-4174-a15a-1b67f0dfd799-kube-api-access-vrrk4\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.182371 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.182517 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82e2f4b7-3511-4f83-bb59-06416fd961ae-host-slash\") pod \"iptables-alerter-fk4ld\" (UID: \"82e2f4b7-3511-4f83-bb59-06416fd961ae\") " pod="openshift-network-operator/iptables-alerter-fk4ld" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.182565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmhjh\" (UniqueName: \"kubernetes.io/projected/82e2f4b7-3511-4f83-bb59-06416fd961ae-kube-api-access-fmhjh\") pod \"iptables-alerter-fk4ld\" (UID: \"82e2f4b7-3511-4f83-bb59-06416fd961ae\") " pod="openshift-network-operator/iptables-alerter-fk4ld" Jul 15 21:29:21.262256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.182588 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82e2f4b7-3511-4f83-bb59-06416fd961ae-host-slash\") pod \"iptables-alerter-fk4ld\" (UID: \"82e2f4b7-3511-4f83-bb59-06416fd961ae\") " pod="openshift-network-operator/iptables-alerter-fk4ld" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.182592 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.182632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/09c29bbd-d242-45cc-b652-d1111a3f417e-ovnkube-config\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.182664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/09c29bbd-d242-45cc-b652-d1111a3f417e-ovnkube-script-lib\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.182701 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-var-lib-kubelet\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.183140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dfb01554-4420-4174-a15a-1b67f0dfd799-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.183190 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-modprobe-d\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.183221 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-tuned\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.183315 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dfb01554-4420-4174-a15a-1b67f0dfd799-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.183342 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/09c29bbd-d242-45cc-b652-d1111a3f417e-ovnkube-config\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.183395 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dfb01554-4420-4174-a15a-1b67f0dfd799-cni-binary-copy\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.183438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-socket-dir-parent\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.183541 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-daemon-config\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.183587 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-system-cni-dir\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184046 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-registration-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184115 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfb01554-4420-4174-a15a-1b67f0dfd799-system-cni-dir\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184126 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-sys\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.263824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-slash\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184178 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-node-log\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-run-ovn-kubernetes\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.184218 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs8b4\" (UniqueName: \"kubernetes.io/projected/09c29bbd-d242-45cc-b652-d1111a3f417e-kube-api-access-xs8b4\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184253 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/82e2f4b7-3511-4f83-bb59-06416fd961ae-iptables-alerter-script\") pod \"iptables-alerter-fk4ld\" (UID: \"82e2f4b7-3511-4f83-bb59-06416fd961ae\") " pod="openshift-network-operator/iptables-alerter-fk4ld" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.184275 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs podName:50eee8aa-8304-424c-8525-91b270d118ac nodeName:}" failed. No retries permitted until 2026-07-15 21:29:21.684255935 +0000 UTC m=+3.187691243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs") pod "network-metrics-daemon-j7s97" (UID: "50eee8aa-8304-424c-8525-91b270d118ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184338 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/09c29bbd-d242-45cc-b652-d1111a3f417e-ovnkube-script-lib\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184396 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-modprobe-d\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184519 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dfb01554-4420-4174-a15a-1b67f0dfd799-cni-binary-copy\") pod \"multus-additional-cni-plugins-rfpjx\" (UID: \"dfb01554-4420-4174-a15a-1b67f0dfd799\") " pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184570 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-daemon-config\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184634 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-slash\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184063 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/319df9cd-0ae2-4abf-8da6-ca519d42834e-registration-dir\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184712 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-var-lib-kubelet\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184723 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-run-ovn-kubernetes\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184766 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-sys\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184770 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-node-log\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.264945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184785 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/82e2f4b7-3511-4f83-bb59-06416fd961ae-iptables-alerter-script\") pod \"iptables-alerter-fk4ld\" (UID: \"82e2f4b7-3511-4f83-bb59-06416fd961ae\") " pod="openshift-network-operator/iptables-alerter-fk4ld" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184822 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-socket-dir-parent\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-cni-dir\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184858 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-os-release\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-conf-dir\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184909 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-kubelet\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184933 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-run-systemd\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-var-lib-kubelet\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.184981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-sysctl-d\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185021 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-run\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czx5\" (UniqueName: \"kubernetes.io/projected/50eee8aa-8304-424c-8525-91b270d118ac-kube-api-access-5czx5\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185095 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-cni-dir\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185098 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-run-ovn\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185142 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-log-socket\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185147 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-kubelet\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-run-netns\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185214 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-run-systemd\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185226 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkdd9\" (UniqueName: \"kubernetes.io/projected/319df9cd-0ae2-4abf-8da6-ca519d42834e-kube-api-access-fkdd9\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.267761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185264 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-var-lib-kubelet\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185291 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-systemd\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-run-netns\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185344 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-cnibin\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-var-lib-cni-bin\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-sysctl-conf\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185472 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-os-release\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185524 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-log-socket\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185564 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-run-netns\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185574 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-sysctl-conf\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185624 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-systemd\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185664 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-run-ovn\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185702 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09c29bbd-d242-45cc-b652-d1111a3f417e-host-run-netns\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185720 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-sysctl-d\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185758 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-cnibin\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185783 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a864031f-a26e-40bb-b130-6d0fe02a72dc-run\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185006 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-multus-conf-dir\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.185804 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ced8f5f2-4875-4a56-bd5d-03ac678d70ed-host-var-lib-cni-bin\") pod \"multus-lbm84\" (UID: \"ced8f5f2-4875-4a56-bd5d-03ac678d70ed\") " pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.268841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.187347 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a864031f-a26e-40bb-b130-6d0fe02a72dc-etc-tuned\") pod \"tuned-bl2b4\" (UID: \"a864031f-a26e-40bb-b130-6d0fe02a72dc\") " pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.269792 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.192687 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmhjh\" (UniqueName: \"kubernetes.io/projected/82e2f4b7-3511-4f83-bb59-06416fd961ae-kube-api-access-fmhjh\") pod \"iptables-alerter-fk4ld\" (UID: \"82e2f4b7-3511-4f83-bb59-06416fd961ae\") " pod="openshift-network-operator/iptables-alerter-fk4ld" Jul 15 21:29:21.269792 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.195012 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs8b4\" (UniqueName: \"kubernetes.io/projected/09c29bbd-d242-45cc-b652-d1111a3f417e-kube-api-access-xs8b4\") pod \"ovnkube-node-hqhrr\" (UID: \"09c29bbd-d242-45cc-b652-d1111a3f417e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.269792 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.195961 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkdd9\" (UniqueName: \"kubernetes.io/projected/319df9cd-0ae2-4abf-8da6-ca519d42834e-kube-api-access-fkdd9\") pod \"aws-ebs-csi-driver-node-gxbvj\" (UID: \"319df9cd-0ae2-4abf-8da6-ca519d42834e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.269792 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.196992 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5czx5\" (UniqueName: \"kubernetes.io/projected/50eee8aa-8304-424c-8525-91b270d118ac-kube-api-access-5czx5\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.284523 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-822j5" Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.294128 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nbzd8" Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:21.298900 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf191e06a_8191_4762_8e60_52d29e065cb2.slice/crio-78e88e20d3584e0050fc9e67ca8d1d1b80b70a074efcbd8c723ef13cd32420f2 WatchSource:0}: Error finding container 78e88e20d3584e0050fc9e67ca8d1d1b80b70a074efcbd8c723ef13cd32420f2: Status 404 returned error can't find the container with id 78e88e20d3584e0050fc9e67ca8d1d1b80b70a074efcbd8c723ef13cd32420f2 Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.303884 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:21.306073 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d05dda3_b68c_4486_9369_a0615903e606.slice/crio-53b0dbb4ac4334da471297ddb177fecb0de36ed8eca83c08bf875b68407ae30c WatchSource:0}: Error finding container 53b0dbb4ac4334da471297ddb177fecb0de36ed8eca83c08bf875b68407ae30c: Status 404 returned error can't find the container with id 53b0dbb4ac4334da471297ddb177fecb0de36ed8eca83c08bf875b68407ae30c Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:21.316597 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c29bbd_d242_45cc_b652_d1111a3f417e.slice/crio-8df61e74694a29f167ba848c97e95b8dc00e133a0e06592ef12ebcaaafd15430 WatchSource:0}: Error finding container 8df61e74694a29f167ba848c97e95b8dc00e133a0e06592ef12ebcaaafd15430: Status 404 returned error can't find the container with id 8df61e74694a29f167ba848c97e95b8dc00e133a0e06592ef12ebcaaafd15430 Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.344396 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fk4ld" Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.353076 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:21.355387 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e2f4b7_3511_4f83_bb59_06416fd961ae.slice/crio-e4f6142353106fe024de18b9012ea7198d2774019e13542879f41e7226b223f7 WatchSource:0}: Error finding container e4f6142353106fe024de18b9012ea7198d2774019e13542879f41e7226b223f7: Status 404 returned error can't find the container with id e4f6142353106fe024de18b9012ea7198d2774019e13542879f41e7226b223f7 Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.362073 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:21.362871 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd733c3ef_eaf2_44b5_8e84_c5f17c26665a.slice/crio-6af377d722482c9076f1d5965868d69ca076cf7481bf7ba9c728788728bf54c0 WatchSource:0}: Error finding container 6af377d722482c9076f1d5965868d69ca076cf7481bf7ba9c728788728bf54c0: Status 404 returned error can't find the container with id 6af377d722482c9076f1d5965868d69ca076cf7481bf7ba9c728788728bf54c0 Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:21.372786 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod319df9cd_0ae2_4abf_8da6_ca519d42834e.slice/crio-c413c884c729d3498b058941e2fe0694dc28708c085d286a06a3d88da4ae6a26 WatchSource:0}: Error finding container c413c884c729d3498b058941e2fe0694dc28708c085d286a06a3d88da4ae6a26: Status 404 returned error can't find the container with id c413c884c729d3498b058941e2fe0694dc28708c085d286a06a3d88da4ae6a26 Jul 15 21:29:21.395065 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.381503 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lbm84" Jul 15 21:29:21.547849 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.396584 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" Jul 15 21:29:21.547849 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:21.398549 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced8f5f2_4875_4a56_bd5d_03ac678d70ed.slice/crio-08f0909152dbd40e506637012a87fcdcb09b8f9b0cfbf3947a1afc84f8cbb754 WatchSource:0}: Error finding container 08f0909152dbd40e506637012a87fcdcb09b8f9b0cfbf3947a1afc84f8cbb754: Status 404 returned error can't find the container with id 08f0909152dbd40e506637012a87fcdcb09b8f9b0cfbf3947a1afc84f8cbb754 Jul 15 21:29:21.547849 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.398644 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rfpjx" Jul 15 21:29:21.689506 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.689470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7zmr\" (UniqueName: \"kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr\") pod \"network-check-target-t9g8r\" (UID: \"0037cbd3-6cda-4518-a2b0-db580fa02577\") " pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:21.689698 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.689519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:21.689698 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.689559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:21.689698 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.689661 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:29:21.689698 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.689675 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:21.689698 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.689680 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:21.689698 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.689689 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:29:21.689698 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.689703 2578 projected.go:196] Error preparing data for projected volume kube-api-access-d7zmr for pod openshift-network-diagnostics/network-check-target-t9g8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:21.690022 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.689736 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs podName:50eee8aa-8304-424c-8525-91b270d118ac nodeName:}" failed. No retries permitted until 2026-07-15 21:29:22.689720189 +0000 UTC m=+4.193155476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs") pod "network-metrics-daemon-j7s97" (UID: "50eee8aa-8304-424c-8525-91b270d118ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:21.690022 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.689760 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret podName:52934511-13c5-4254-bdbd-b3e67a77c80f nodeName:}" failed. No retries permitted until 2026-07-15 21:29:22.689742562 +0000 UTC m=+4.193177837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret") pod "global-pull-secret-syncer-6jqqs" (UID: "52934511-13c5-4254-bdbd-b3e67a77c80f") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:21.690022 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:21.689780 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr podName:0037cbd3-6cda-4518-a2b0-db580fa02577 nodeName:}" failed. No retries permitted until 2026-07-15 21:29:22.68977033 +0000 UTC m=+4.193205610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7zmr" (UniqueName: "kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr") pod "network-check-target-t9g8r" (UID: "0037cbd3-6cda-4518-a2b0-db580fa02577") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:21.982378 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.982276 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 21:24:21 +0000 UTC" deadline="2028-02-19 04:58:27.216314898 +0000 UTC" Jul 15 21:29:21.982378 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:21.982350 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13999h29m5.233976526s" Jul 15 21:29:22.059238 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.059190 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" event={"ID":"a864031f-a26e-40bb-b130-6d0fe02a72dc","Type":"ContainerStarted","Data":"9d9508af9cd1b8f19ada4817a350a58b7e9b17fcf0ed240249120b225b7d6650"} Jul 15 21:29:22.060961 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.060924 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lbm84" event={"ID":"ced8f5f2-4875-4a56-bd5d-03ac678d70ed","Type":"ContainerStarted","Data":"08f0909152dbd40e506637012a87fcdcb09b8f9b0cfbf3947a1afc84f8cbb754"} Jul 15 21:29:22.062658 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.062516 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" event={"ID":"319df9cd-0ae2-4abf-8da6-ca519d42834e","Type":"ContainerStarted","Data":"c413c884c729d3498b058941e2fe0694dc28708c085d286a06a3d88da4ae6a26"} Jul 15 21:29:22.064510 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.064439 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lbd68" event={"ID":"d733c3ef-eaf2-44b5-8e84-c5f17c26665a","Type":"ContainerStarted","Data":"6af377d722482c9076f1d5965868d69ca076cf7481bf7ba9c728788728bf54c0"} Jul 15 21:29:22.066147 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.066056 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fk4ld" event={"ID":"82e2f4b7-3511-4f83-bb59-06416fd961ae","Type":"ContainerStarted","Data":"e4f6142353106fe024de18b9012ea7198d2774019e13542879f41e7226b223f7"} Jul 15 21:29:22.068616 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.068501 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" event={"ID":"09c29bbd-d242-45cc-b652-d1111a3f417e","Type":"ContainerStarted","Data":"8df61e74694a29f167ba848c97e95b8dc00e133a0e06592ef12ebcaaafd15430"} Jul 15 21:29:22.070919 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.070872 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nbzd8" event={"ID":"4d05dda3-b68c-4486-9369-a0615903e606","Type":"ContainerStarted","Data":"53b0dbb4ac4334da471297ddb177fecb0de36ed8eca83c08bf875b68407ae30c"} Jul 15 21:29:22.072437 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.072408 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-822j5" event={"ID":"f191e06a-8191-4762-8e60-52d29e065cb2","Type":"ContainerStarted","Data":"78e88e20d3584e0050fc9e67ca8d1d1b80b70a074efcbd8c723ef13cd32420f2"} Jul 15 21:29:22.697945 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.697899 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:22.698187 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.698000 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7zmr\" (UniqueName: \"kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr\") pod \"network-check-target-t9g8r\" (UID: \"0037cbd3-6cda-4518-a2b0-db580fa02577\") " pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:22.698187 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.698038 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:22.698320 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:22.698198 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:22.698320 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:22.698264 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret podName:52934511-13c5-4254-bdbd-b3e67a77c80f nodeName:}" failed. No retries permitted until 2026-07-15 21:29:24.698244734 +0000 UTC m=+6.201680024 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret") pod "global-pull-secret-syncer-6jqqs" (UID: "52934511-13c5-4254-bdbd-b3e67a77c80f") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:22.698720 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:22.698698 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:22.698805 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:22.698764 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs podName:50eee8aa-8304-424c-8525-91b270d118ac nodeName:}" failed. No retries permitted until 2026-07-15 21:29:24.698747247 +0000 UTC m=+6.202182528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs") pod "network-metrics-daemon-j7s97" (UID: "50eee8aa-8304-424c-8525-91b270d118ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:22.698870 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:22.698844 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:29:22.698870 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:22.698859 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:29:22.698975 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:22.698873 2578 projected.go:196] Error preparing data for projected volume kube-api-access-d7zmr for pod openshift-network-diagnostics/network-check-target-t9g8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:22.698975 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:22.698907 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr podName:0037cbd3-6cda-4518-a2b0-db580fa02577 nodeName:}" failed. No retries permitted until 2026-07-15 21:29:24.698895423 +0000 UTC m=+6.202330702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7zmr" (UniqueName: "kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr") pod "network-check-target-t9g8r" (UID: "0037cbd3-6cda-4518-a2b0-db580fa02577") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:22.983662 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.983561 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 21:24:21 +0000 UTC" deadline="2028-03-10 16:48:40.739507794 +0000 UTC" Jul 15 21:29:22.983662 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:22.983607 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14491h19m17.755906164s" Jul 15 21:29:23.048885 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:23.047983 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:23.048885 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:23.048136 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:23.048885 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:23.048591 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:23.048885 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:23.048698 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:23.048885 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:23.048752 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:23.048885 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:23.048808 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:24.081949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:24.081845 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfpjx" event={"ID":"dfb01554-4420-4174-a15a-1b67f0dfd799","Type":"ContainerStarted","Data":"dd57f84956398d97a2c8572b53d24cd6624ba4a2d32c344fec4574ce98a88a28"} Jul 15 21:29:24.716289 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:24.716247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7zmr\" (UniqueName: \"kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr\") pod \"network-check-target-t9g8r\" (UID: \"0037cbd3-6cda-4518-a2b0-db580fa02577\") " pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:24.716489 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:24.716304 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:24.716489 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:24.716345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:24.716489 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:24.716459 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:24.716489 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:24.716462 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:29:24.716489 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:24.716491 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:29:24.716789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:24.716506 2578 projected.go:196] Error preparing data for projected volume kube-api-access-d7zmr for pod openshift-network-diagnostics/network-check-target-t9g8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:24.716789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:24.716524 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs podName:50eee8aa-8304-424c-8525-91b270d118ac nodeName:}" failed. No retries permitted until 2026-07-15 21:29:28.716503832 +0000 UTC m=+10.219939110 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs") pod "network-metrics-daemon-j7s97" (UID: "50eee8aa-8304-424c-8525-91b270d118ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:24.716789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:24.716562 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr podName:0037cbd3-6cda-4518-a2b0-db580fa02577 nodeName:}" failed. No retries permitted until 2026-07-15 21:29:28.716544955 +0000 UTC m=+10.219980246 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7zmr" (UniqueName: "kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr") pod "network-check-target-t9g8r" (UID: "0037cbd3-6cda-4518-a2b0-db580fa02577") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:24.716789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:24.716633 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:24.716789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:24.716668 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret podName:52934511-13c5-4254-bdbd-b3e67a77c80f nodeName:}" failed. No retries permitted until 2026-07-15 21:29:28.716656513 +0000 UTC m=+10.220091800 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret") pod "global-pull-secret-syncer-6jqqs" (UID: "52934511-13c5-4254-bdbd-b3e67a77c80f") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:25.046457 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:25.046292 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:25.046622 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:25.046461 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:25.046622 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:25.046473 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:25.046622 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:25.046502 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:25.046622 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:25.046613 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:25.046821 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:25.046707 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:27.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:27.044675 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:27.045055 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:27.044816 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:27.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:27.044828 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:27.045055 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:27.044917 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:27.045055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:27.044675 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:27.045055 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:27.045009 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:28.752328 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:28.752280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7zmr\" (UniqueName: \"kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr\") pod \"network-check-target-t9g8r\" (UID: \"0037cbd3-6cda-4518-a2b0-db580fa02577\") " pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:28.752789 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:28.752346 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:28.752789 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:28.752388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:28.752789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:28.752497 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:29:28.752789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:28.752520 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:29:28.752789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:28.752522 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:28.752789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:28.752531 2578 projected.go:196] Error preparing data for projected volume kube-api-access-d7zmr for pod openshift-network-diagnostics/network-check-target-t9g8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:28.752789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:28.752529 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:28.752789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:28.752579 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr podName:0037cbd3-6cda-4518-a2b0-db580fa02577 nodeName:}" failed. No retries permitted until 2026-07-15 21:29:36.752565227 +0000 UTC m=+18.256000503 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7zmr" (UniqueName: "kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr") pod "network-check-target-t9g8r" (UID: "0037cbd3-6cda-4518-a2b0-db580fa02577") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:28.752789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:28.752604 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret podName:52934511-13c5-4254-bdbd-b3e67a77c80f nodeName:}" failed. No retries permitted until 2026-07-15 21:29:36.752587975 +0000 UTC m=+18.256023265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret") pod "global-pull-secret-syncer-6jqqs" (UID: "52934511-13c5-4254-bdbd-b3e67a77c80f") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:28.752789 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:28.752665 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs podName:50eee8aa-8304-424c-8525-91b270d118ac nodeName:}" failed. No retries permitted until 2026-07-15 21:29:36.752650794 +0000 UTC m=+18.256086082 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs") pod "network-metrics-daemon-j7s97" (UID: "50eee8aa-8304-424c-8525-91b270d118ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:29.049364 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:29.047602 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:29.049364 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:29.047774 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:29.049364 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:29.048268 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:29.049364 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:29.048421 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:29.049364 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:29.048459 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:29.049364 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:29.048588 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:31.045786 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:31.045347 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:31.045786 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:31.045347 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:31.045786 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:31.045499 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:31.045786 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:31.045559 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:31.045786 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:31.045585 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:31.045786 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:31.045649 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:33.045728 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:33.045687 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:33.046175 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:33.045687 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:33.046175 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:33.045687 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:33.046175 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:33.045873 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:33.046175 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:33.045849 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:33.046175 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:33.045967 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:35.045369 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:35.045324 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:35.045825 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:35.045324 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:35.045825 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:35.045474 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:35.045825 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:35.045555 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:35.045825 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:35.045324 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:35.045825 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:35.045644 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:36.814147 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:36.814106 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:36.814593 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:36.814191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7zmr\" (UniqueName: \"kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr\") pod \"network-check-target-t9g8r\" (UID: \"0037cbd3-6cda-4518-a2b0-db580fa02577\") " pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:36.814593 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:36.814225 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:36.814593 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:36.814195 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:36.814593 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:36.814273 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:29:36.814593 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:36.814291 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:29:36.814593 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:36.814303 2578 projected.go:196] Error preparing data for projected volume kube-api-access-d7zmr for pod openshift-network-diagnostics/network-check-target-t9g8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:36.814593 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:36.814303 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:36.814593 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:36.814356 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr podName:0037cbd3-6cda-4518-a2b0-db580fa02577 nodeName:}" failed. No retries permitted until 2026-07-15 21:29:52.814343873 +0000 UTC m=+34.317779150 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7zmr" (UniqueName: "kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr") pod "network-check-target-t9g8r" (UID: "0037cbd3-6cda-4518-a2b0-db580fa02577") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:36.814593 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:36.814415 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs podName:50eee8aa-8304-424c-8525-91b270d118ac nodeName:}" failed. No retries permitted until 2026-07-15 21:29:52.81439049 +0000 UTC m=+34.317825767 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs") pod "network-metrics-daemon-j7s97" (UID: "50eee8aa-8304-424c-8525-91b270d118ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:36.814593 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:36.814433 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret podName:52934511-13c5-4254-bdbd-b3e67a77c80f nodeName:}" failed. No retries permitted until 2026-07-15 21:29:52.814423804 +0000 UTC m=+34.317859080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret") pod "global-pull-secret-syncer-6jqqs" (UID: "52934511-13c5-4254-bdbd-b3e67a77c80f") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:37.045695 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:37.045657 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:37.045883 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:37.045657 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:37.045883 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:37.045812 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:37.045883 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:37.045667 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:37.046049 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:37.045908 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:37.046049 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:37.045958 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:39.046158 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:39.046118 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:39.046545 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:39.046227 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:39.046545 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:39.046240 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:39.046545 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:39.046314 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:39.046545 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:39.046372 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:39.046545 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:39.046439 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:41.045156 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:41.044937 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:41.045521 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:41.044950 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:41.045521 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:41.045278 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:41.045521 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:41.045334 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:41.045521 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:41.044996 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:41.045521 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:41.045431 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:41.133030 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:41.132993 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" event={"ID":"a864031f-a26e-40bb-b130-6d0fe02a72dc","Type":"ContainerStarted","Data":"9eca5db8d39fb45df66f0fbb0f1721a3b38cb28f0e3a0dfa8d8108accdc74209"} Jul 15 21:29:41.135300 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:41.135276 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lbm84" event={"ID":"ced8f5f2-4875-4a56-bd5d-03ac678d70ed","Type":"ContainerStarted","Data":"22b78b20a424871686aaf4aa2e75d5c09117aa761626eb4071209e93525f1a1d"} Jul 15 21:29:41.136530 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:41.136509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-62.ec2.internal" event={"ID":"40d5eee52fbcaf2e5bbfd3cc3c45fbae","Type":"ContainerStarted","Data":"37da40020fa38f4585c0a587b7aaebe38b8fea8aa8a7b3f40df62b8094835a32"} Jul 15 21:29:41.149552 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:41.149507 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-62.ec2.internal" podStartSLOduration=20.14949488 podStartE2EDuration="20.14949488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:41.149349749 +0000 UTC m=+22.652785045" watchObservedRunningTime="2026-07-15 21:29:41.14949488 +0000 UTC m=+22.652930253" Jul 15 21:29:42.140254 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.140052 2578 generic.go:356] "Generic (PLEG): container finished" podID="cc89e18af835f63bc098b914e5f0f33d" containerID="241054048d550847e7e7c4d5f62a785e9c9677e837e12872d744fd560f21f36e" exitCode=0 Jul 15 21:29:42.141066 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.140146 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" event={"ID":"cc89e18af835f63bc098b914e5f0f33d","Type":"ContainerDied","Data":"241054048d550847e7e7c4d5f62a785e9c9677e837e12872d744fd560f21f36e"} Jul 15 21:29:42.141903 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.141870 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" event={"ID":"319df9cd-0ae2-4abf-8da6-ca519d42834e","Type":"ContainerStarted","Data":"cd2b1fd46b838e55942aa5c412a4aac16e88e84d0f55daf32305d2d7541450c1"} Jul 15 21:29:42.143252 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.143232 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lbd68" event={"ID":"d733c3ef-eaf2-44b5-8e84-c5f17c26665a","Type":"ContainerStarted","Data":"c63688f26f8c96afcbb8eecbdb153e4bdc1c1d989d596fcfe2e5d4d3388e95ee"} Jul 15 21:29:42.144706 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.144686 2578 generic.go:356] "Generic (PLEG): container finished" podID="dfb01554-4420-4174-a15a-1b67f0dfd799" containerID="baa50781666bd1436d3eaed34ce79c4edebd6c3e41f947eb29fc88cabfe616c4" exitCode=0 Jul 15 21:29:42.144786 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.144757 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfpjx" event={"ID":"dfb01554-4420-4174-a15a-1b67f0dfd799","Type":"ContainerDied","Data":"baa50781666bd1436d3eaed34ce79c4edebd6c3e41f947eb29fc88cabfe616c4"} Jul 15 21:29:42.147349 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.147333 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:29:42.147657 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.147642 2578 generic.go:356] "Generic (PLEG): container finished" podID="09c29bbd-d242-45cc-b652-d1111a3f417e" containerID="b3070a4328afe7e8732de0f1653be63cae9390c2a9242d185e21b8fa2d60fa6c" exitCode=1 Jul 15 21:29:42.147721 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.147703 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" event={"ID":"09c29bbd-d242-45cc-b652-d1111a3f417e","Type":"ContainerStarted","Data":"44075fd31ef573587191b16ad57284c91124c8125bf73d1baccb5cb5f0f99bed"} Jul 15 21:29:42.147788 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.147769 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" event={"ID":"09c29bbd-d242-45cc-b652-d1111a3f417e","Type":"ContainerStarted","Data":"2e5272dfa34c9c649bb73bcc0cb654ad14c1145e7a580ed0c237b10a39c362ca"} Jul 15 21:29:42.147886 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.147810 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" event={"ID":"09c29bbd-d242-45cc-b652-d1111a3f417e","Type":"ContainerStarted","Data":"3161613df0915e044c222561eab4d09d7257dba7552d2724cb6f697f44fe1005"} Jul 15 21:29:42.147886 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.147853 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" event={"ID":"09c29bbd-d242-45cc-b652-d1111a3f417e","Type":"ContainerStarted","Data":"3a4232e314970d1a9a62952dd9b5d20f955310fe47b604cd37e4d88e50c39aa6"} Jul 15 21:29:42.147991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.147892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" event={"ID":"09c29bbd-d242-45cc-b652-d1111a3f417e","Type":"ContainerDied","Data":"b3070a4328afe7e8732de0f1653be63cae9390c2a9242d185e21b8fa2d60fa6c"} Jul 15 21:29:42.147991 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.147940 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" event={"ID":"09c29bbd-d242-45cc-b652-d1111a3f417e","Type":"ContainerStarted","Data":"3ffa5adb9a13e88a781a369223a00f5158c08c43b8fd9f7d143f8428ddfc3c64"} Jul 15 21:29:42.148892 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.148875 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nbzd8" event={"ID":"4d05dda3-b68c-4486-9369-a0615903e606","Type":"ContainerStarted","Data":"4eaea42401e97748a36dd4fd45048034280daa5b81e8dbb5a114294528e1fc70"} Jul 15 21:29:42.150261 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.150216 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-822j5" event={"ID":"f191e06a-8191-4762-8e60-52d29e065cb2","Type":"ContainerStarted","Data":"c69382bb20e75971269725116c969004e4a186bd5e480be2d7aafe0ee1c63a43"} Jul 15 21:29:42.194428 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.194196 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-822j5" podStartSLOduration=3.989158566 podStartE2EDuration="23.194180165s" totalImagesPullingTime="19.205021599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:42.194119471 +0000 UTC m=+23.697554767" watchObservedRunningTime="2026-07-15 21:29:42.194180165 +0000 UTC m=+23.697615479" Jul 15 21:29:42.224731 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.224377 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nbzd8" podStartSLOduration=4.026434295 podStartE2EDuration="23.224358972s" totalImagesPullingTime="19.197924677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:42.224118114 +0000 UTC m=+23.727553409" watchObservedRunningTime="2026-07-15 21:29:42.224358972 +0000 UTC m=+23.727794268" Jul 15 21:29:42.224731 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.224621 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lbd68" podStartSLOduration=4.081909715 podStartE2EDuration="23.224611964s" totalImagesPullingTime="19.142702249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:42.209132341 +0000 UTC m=+23.712567629" watchObservedRunningTime="2026-07-15 21:29:42.224611964 +0000 UTC m=+23.728047258" Jul 15 21:29:42.243358 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.243298 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lbm84" podStartSLOduration=3.745675271 podStartE2EDuration="23.243279988s" totalImagesPullingTime="19.497604717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:42.242390741 +0000 UTC m=+23.745826036" watchObservedRunningTime="2026-07-15 21:29:42.243279988 +0000 UTC m=+23.746715285" Jul 15 21:29:42.258408 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:42.258369 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bl2b4" podStartSLOduration=4.184744141 podStartE2EDuration="23.258354583s" totalImagesPullingTime="19.073610442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:42.257817655 +0000 UTC m=+23.761252953" watchObservedRunningTime="2026-07-15 21:29:42.258354583 +0000 UTC m=+23.761789876" Jul 15 21:29:43.045435 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:43.045399 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:43.045804 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:43.045399 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:43.045804 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:43.045538 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:43.045804 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:43.045399 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:43.045804 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:43.045594 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:43.045804 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:43.045655 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:43.154495 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:43.154458 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fk4ld" event={"ID":"82e2f4b7-3511-4f83-bb59-06416fd961ae","Type":"ContainerStarted","Data":"c2581c0bd85ba38f6a988fe5725f1b8feab0b3a6661ee82516a40afafd6db79d"} Jul 15 21:29:43.156575 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:43.156498 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" event={"ID":"cc89e18af835f63bc098b914e5f0f33d","Type":"ContainerStarted","Data":"06feadc9d5d2fa1af6fe2210a54746ad363b50e3e3cc9e32a5d5c53df14c84b4"} Jul 15 21:29:43.173402 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:43.173355 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fk4ld" podStartSLOduration=4.671673522 podStartE2EDuration="24.173338492s" totalImagesPullingTime="19.50166497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:43.173324199 +0000 UTC m=+24.676759495" watchObservedRunningTime="2026-07-15 21:29:43.173338492 +0000 UTC m=+24.676773788" Jul 15 21:29:43.187915 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:43.187862 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-62.ec2.internal" podStartSLOduration=22.187848232 podStartE2EDuration="22.187848232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:43.187740385 +0000 UTC m=+24.691175680" watchObservedRunningTime="2026-07-15 21:29:43.187848232 +0000 UTC m=+24.691283528" Jul 15 21:29:43.923750 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:43.923698 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 21:29:44.018870 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:44.018765 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T21:29:43.923724691Z","UUID":"6201bc31-5bc6-4697-8d83-5fccaa56de30","Handler":null,"Name":"","Endpoint":""} Jul 15 21:29:44.020780 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:44.020753 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 21:29:44.020891 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:44.020798 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 21:29:44.033498 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:44.033473 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:44.034024 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:44.034008 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:44.160844 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:44.160806 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" event={"ID":"319df9cd-0ae2-4abf-8da6-ca519d42834e","Type":"ContainerStarted","Data":"427d718f9a95d8142590adff781bd3e1c33169e9a4e59b29c915e57ed26097c2"} Jul 15 21:29:44.164154 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:44.164122 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:29:44.164503 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:44.164476 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" event={"ID":"09c29bbd-d242-45cc-b652-d1111a3f417e","Type":"ContainerStarted","Data":"fff10c967d55880f41569996770d5774c158f836e7230c8bbdf0c7fdedec4e6f"} Jul 15 21:29:45.045835 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:45.045799 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:45.046016 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:45.045799 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:45.046016 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:45.045945 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:45.046016 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:45.045957 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:45.046175 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:45.046072 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:45.046238 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:45.046210 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:45.166286 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:45.166261 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 21:29:46.875643 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:46.875600 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:46.876387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:46.875719 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 21:29:46.876387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:46.876173 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lbd68" Jul 15 21:29:46.948548 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:46.948470 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:47.036546 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.036512 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:47.045442 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.045419 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:47.045524 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.045427 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:47.045567 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:47.045533 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:47.045609 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:47.045590 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:47.045643 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.045427 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:47.045678 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:47.045662 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:47.172694 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.172656 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" event={"ID":"319df9cd-0ae2-4abf-8da6-ca519d42834e","Type":"ContainerStarted","Data":"952e102095139d8beff5bde5a3e871d74f441bd34ac3e1d10f4dc5b79a5b6900"} Jul 15 21:29:47.174311 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.174283 2578 generic.go:356] "Generic (PLEG): container finished" podID="dfb01554-4420-4174-a15a-1b67f0dfd799" containerID="4afcdd5af0ebd71b40548cd3396e3ecab2ea7a0327847971e86676f99ee65aad" exitCode=0 Jul 15 21:29:47.174424 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.174357 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfpjx" event={"ID":"dfb01554-4420-4174-a15a-1b67f0dfd799","Type":"ContainerDied","Data":"4afcdd5af0ebd71b40548cd3396e3ecab2ea7a0327847971e86676f99ee65aad"} Jul 15 21:29:47.178024 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.178008 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:29:47.178418 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.178400 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" event={"ID":"09c29bbd-d242-45cc-b652-d1111a3f417e","Type":"ContainerStarted","Data":"c6421847a894a1a62cf0e733daa48f5c60699a6216397ff15d3d429b079979e9"} Jul 15 21:29:47.178568 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.178552 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:47.178753 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.178741 2578 scope.go:117] "RemoveContainer" containerID="b3070a4328afe7e8732de0f1653be63cae9390c2a9242d185e21b8fa2d60fa6c" Jul 15 21:29:47.194414 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:47.193729 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxbvj" podStartSLOduration=3.635884418 podStartE2EDuration="28.193709012s" totalImagesPullingTime="24.557824594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:47.190442972 +0000 UTC m=+28.693878266" watchObservedRunningTime="2026-07-15 21:29:47.193709012 +0000 UTC m=+28.697144307" Jul 15 21:29:48.184874 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.184841 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:29:48.185397 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.185372 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" event={"ID":"09c29bbd-d242-45cc-b652-d1111a3f417e","Type":"ContainerStarted","Data":"ca7ca04b02b4c967f4e27664f31d6a63bfc8c48a197e8dcf09ec52c72ac6da3a"} Jul 15 21:29:48.185706 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.185686 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:48.185804 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.185763 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:48.202720 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.202694 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:48.202875 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.202809 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:29:48.212158 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.212040 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" podStartSLOduration=9.336137647 podStartE2EDuration="29.212018706s" totalImagesPullingTime="19.875881059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:48.210387676 +0000 UTC m=+29.713822971" watchObservedRunningTime="2026-07-15 21:29:48.212018706 +0000 UTC m=+29.715454002" Jul 15 21:29:48.446921 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.446834 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:48.457581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.457554 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j7s97"] Jul 15 21:29:48.457700 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.457667 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:48.457768 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:48.457752 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:48.462411 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.462382 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t9g8r"] Jul 15 21:29:48.462528 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.462508 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:48.462631 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:48.462611 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:48.462974 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.462954 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6jqqs"] Jul 15 21:29:48.463054 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.463044 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:48.463157 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:48.463140 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:48.543564 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:48.543531 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:49.189141 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:49.189105 2578 generic.go:356] "Generic (PLEG): container finished" podID="dfb01554-4420-4174-a15a-1b67f0dfd799" containerID="40a60e9fa6fe3aa13c936f9d3bef8eb64df5ca640f67a9756c53efee1dcd8547" exitCode=0 Jul 15 21:29:49.189596 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:49.189190 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfpjx" event={"ID":"dfb01554-4420-4174-a15a-1b67f0dfd799","Type":"ContainerDied","Data":"40a60e9fa6fe3aa13c936f9d3bef8eb64df5ca640f67a9756c53efee1dcd8547"} Jul 15 21:29:49.317365 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:49.317326 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:49.367588 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:49.367553 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:50.044976 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:50.044946 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:50.045157 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:50.044945 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:50.045157 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:50.045043 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:50.045157 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:50.045148 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:50.045264 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:50.044945 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:50.045264 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:50.045214 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:50.194159 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:50.194050 2578 generic.go:356] "Generic (PLEG): container finished" podID="dfb01554-4420-4174-a15a-1b67f0dfd799" containerID="5e03b4ecf12d20bb7079097ff3722791b4969f20d09351467d90ea12c4177de5" exitCode=0 Jul 15 21:29:50.194159 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:50.194133 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfpjx" event={"ID":"dfb01554-4420-4174-a15a-1b67f0dfd799","Type":"ContainerDied","Data":"5e03b4ecf12d20bb7079097ff3722791b4969f20d09351467d90ea12c4177de5"} Jul 15 21:29:50.346480 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:50.346441 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:50.445695 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:50.445605 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:51.045844 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:51.045591 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:51.098756 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:51.098672 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:52.044887 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:52.044854 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:52.045313 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:52.044900 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:52.045313 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:52.044985 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jqqs" podUID="52934511-13c5-4254-bdbd-b3e67a77c80f" Jul 15 21:29:52.045313 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:52.045032 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7s97" podUID="50eee8aa-8304-424c-8525-91b270d118ac" Jul 15 21:29:52.045313 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:52.045081 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:52.045313 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:52.045199 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9g8r" podUID="0037cbd3-6cda-4518-a2b0-db580fa02577" Jul 15 21:29:52.828878 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:52.828824 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7zmr\" (UniqueName: \"kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr\") pod \"network-check-target-t9g8r\" (UID: \"0037cbd3-6cda-4518-a2b0-db580fa02577\") " pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:52.828878 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:52.828884 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:52.829138 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:52.828924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:52.829138 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:52.828999 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:52.829138 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:52.829018 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:29:52.829138 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:52.829054 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:29:52.829138 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:52.829068 2578 projected.go:196] Error preparing data for projected volume kube-api-access-d7zmr for pod openshift-network-diagnostics/network-check-target-t9g8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:52.829138 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:52.829073 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret podName:52934511-13c5-4254-bdbd-b3e67a77c80f nodeName:}" failed. No retries permitted until 2026-07-15 21:30:24.829054208 +0000 UTC m=+66.332489495 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret") pod "global-pull-secret-syncer-6jqqs" (UID: "52934511-13c5-4254-bdbd-b3e67a77c80f") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:29:52.829138 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:52.829024 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:52.829417 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:52.829135 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr podName:0037cbd3-6cda-4518-a2b0-db580fa02577 nodeName:}" failed. No retries permitted until 2026-07-15 21:30:24.829119518 +0000 UTC m=+66.332554792 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7zmr" (UniqueName: "kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr") pod "network-check-target-t9g8r" (UID: "0037cbd3-6cda-4518-a2b0-db580fa02577") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:29:52.829417 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:52.829179 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs podName:50eee8aa-8304-424c-8525-91b270d118ac nodeName:}" failed. No retries permitted until 2026-07-15 21:30:24.829168452 +0000 UTC m=+66.332603729 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs") pod "network-metrics-daemon-j7s97" (UID: "50eee8aa-8304-424c-8525-91b270d118ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:29:53.794358 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.794330 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-62.ec2.internal" event="NodeReady" Jul 15 21:29:53.794911 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.794469 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 21:29:53.827267 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.827182 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6dcd789698-x8rkr"] Jul 15 21:29:53.859937 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.859897 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ml6fl"] Jul 15 21:29:53.860117 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.860042 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:53.863225 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.863199 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 21:29:53.863225 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.863220 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 21:29:53.863423 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.863293 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 21:29:53.863522 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.863510 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kpvhq\"" Jul 15 21:29:53.873377 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.873355 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 21:29:53.873809 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.873784 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p7x5z"] Jul 15 21:29:53.873927 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.873906 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:53.876729 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.876710 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c96dg\"" Jul 15 21:29:53.876729 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.876724 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 21:29:53.876865 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.876725 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 21:29:53.889863 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.889842 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dcd789698-x8rkr"] Jul 15 21:29:53.889977 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.889877 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p7x5z"] Jul 15 21:29:53.889977 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.889966 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p7x5z" Jul 15 21:29:53.890080 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.889969 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ml6fl"] Jul 15 21:29:53.893763 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.893639 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 21:29:53.894003 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.893982 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5mj78\"" Jul 15 21:29:53.894600 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.894577 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 21:29:53.906984 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.906957 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 21:29:53.937754 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.937717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkt6q\" (UniqueName: \"kubernetes.io/projected/50607b0d-4426-4d4b-97af-4db2e1e305fa-kube-api-access-wkt6q\") pod \"ingress-canary-p7x5z\" (UID: \"50607b0d-4426-4d4b-97af-4db2e1e305fa\") " pod="openshift-ingress-canary/ingress-canary-p7x5z" Jul 15 21:29:53.937754 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.937748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-installation-pull-secrets\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:53.937955 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.937770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/07138502-cc97-44da-9b9f-a0ea92aa4e68-config-volume\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:53.937955 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.937787 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkvxw\" (UniqueName: \"kubernetes.io/projected/07138502-cc97-44da-9b9f-a0ea92aa4e68-kube-api-access-fkvxw\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:53.937955 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.937815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-trusted-ca\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:53.937955 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.937881 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07138502-cc97-44da-9b9f-a0ea92aa4e68-tmp-dir\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:53.937955 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.937921 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:53.938144 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.937958 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/99581639-234d-413f-be81-2ba54826a4cf-ca-trust-extracted\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:53.938144 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.937976 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnnz9\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-kube-api-access-rnnz9\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:53.938144 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.937994 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50607b0d-4426-4d4b-97af-4db2e1e305fa-cert\") pod \"ingress-canary-p7x5z\" (UID: \"50607b0d-4426-4d4b-97af-4db2e1e305fa\") " pod="openshift-ingress-canary/ingress-canary-p7x5z" Jul 15 21:29:53.938144 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.938007 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-bound-sa-token\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:53.938144 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.938048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:53.938144 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.938114 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-image-registry-private-configuration\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:53.938144 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:53.938137 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-registry-certificates\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.039136 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039070 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:54.039136 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039133 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/99581639-234d-413f-be81-2ba54826a4cf-ca-trust-extracted\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.039378 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnnz9\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-kube-api-access-rnnz9\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.039378 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039174 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50607b0d-4426-4d4b-97af-4db2e1e305fa-cert\") pod \"ingress-canary-p7x5z\" (UID: \"50607b0d-4426-4d4b-97af-4db2e1e305fa\") " pod="openshift-ingress-canary/ingress-canary-p7x5z" Jul 15 21:29:54.039378 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-bound-sa-token\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.039378 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039215 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.039378 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:54.039224 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:29:54.039378 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-image-registry-private-configuration\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.039378 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039274 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-registry-certificates\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.039378 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:54.039307 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls podName:07138502-cc97-44da-9b9f-a0ea92aa4e68 nodeName:}" failed. No retries permitted until 2026-07-15 21:29:54.539285352 +0000 UTC m=+36.042720630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls") pod "dns-default-ml6fl" (UID: "07138502-cc97-44da-9b9f-a0ea92aa4e68") : secret "dns-default-metrics-tls" not found Jul 15 21:29:54.039378 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkt6q\" (UniqueName: \"kubernetes.io/projected/50607b0d-4426-4d4b-97af-4db2e1e305fa-kube-api-access-wkt6q\") pod \"ingress-canary-p7x5z\" (UID: \"50607b0d-4426-4d4b-97af-4db2e1e305fa\") " pod="openshift-ingress-canary/ingress-canary-p7x5z" Jul 15 21:29:54.039752 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-installation-pull-secrets\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.039752 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039413 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/07138502-cc97-44da-9b9f-a0ea92aa4e68-config-volume\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:54.039752 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkvxw\" (UniqueName: \"kubernetes.io/projected/07138502-cc97-44da-9b9f-a0ea92aa4e68-kube-api-access-fkvxw\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:54.039752 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-trusted-ca\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.039752 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07138502-cc97-44da-9b9f-a0ea92aa4e68-tmp-dir\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:54.039752 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.039650 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/99581639-234d-413f-be81-2ba54826a4cf-ca-trust-extracted\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.039752 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:54.039363 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:29:54.040125 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:54.039760 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcd789698-x8rkr: secret "image-registry-tls" not found Jul 15 21:29:54.040125 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:54.039812 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls podName:99581639-234d-413f-be81-2ba54826a4cf nodeName:}" failed. No retries permitted until 2026-07-15 21:29:54.5397971 +0000 UTC m=+36.043232393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls") pod "image-registry-6dcd789698-x8rkr" (UID: "99581639-234d-413f-be81-2ba54826a4cf") : secret "image-registry-tls" not found Jul 15 21:29:54.040125 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.040048 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/07138502-cc97-44da-9b9f-a0ea92aa4e68-config-volume\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:54.040125 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.040059 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-registry-certificates\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.040293 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.040249 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07138502-cc97-44da-9b9f-a0ea92aa4e68-tmp-dir\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:54.040559 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.040536 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-trusted-ca\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.043545 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.043523 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-image-registry-private-configuration\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.043657 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.043590 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-installation-pull-secrets\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.043657 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.043607 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50607b0d-4426-4d4b-97af-4db2e1e305fa-cert\") pod \"ingress-canary-p7x5z\" (UID: \"50607b0d-4426-4d4b-97af-4db2e1e305fa\") " pod="openshift-ingress-canary/ingress-canary-p7x5z" Jul 15 21:29:54.044765 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.044748 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:29:54.044808 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.044771 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:29:54.044808 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.044773 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:29:54.048212 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.048192 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 21:29:54.048326 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.048281 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 21:29:54.048476 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.048453 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 21:29:54.048607 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.048590 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-75svw\"" Jul 15 21:29:54.048697 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.048680 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8nwm9\"" Jul 15 21:29:54.048817 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.048795 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkvxw\" (UniqueName: \"kubernetes.io/projected/07138502-cc97-44da-9b9f-a0ea92aa4e68-kube-api-access-fkvxw\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:54.048884 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.048836 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 21:29:54.049117 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.049076 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-bound-sa-token\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.049635 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.049606 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkt6q\" (UniqueName: \"kubernetes.io/projected/50607b0d-4426-4d4b-97af-4db2e1e305fa-kube-api-access-wkt6q\") pod \"ingress-canary-p7x5z\" (UID: \"50607b0d-4426-4d4b-97af-4db2e1e305fa\") " pod="openshift-ingress-canary/ingress-canary-p7x5z" Jul 15 21:29:54.050680 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.050664 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnnz9\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-kube-api-access-rnnz9\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.199530 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.199458 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p7x5z" Jul 15 21:29:54.543260 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.543080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:54.543421 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.543285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:54.543421 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:54.543193 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:29:54.543421 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:54.543385 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:29:54.543421 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:54.543394 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcd789698-x8rkr: secret "image-registry-tls" not found Jul 15 21:29:54.543421 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:54.543398 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls podName:07138502-cc97-44da-9b9f-a0ea92aa4e68 nodeName:}" failed. No retries permitted until 2026-07-15 21:29:55.543379195 +0000 UTC m=+37.046814479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls") pod "dns-default-ml6fl" (UID: "07138502-cc97-44da-9b9f-a0ea92aa4e68") : secret "dns-default-metrics-tls" not found Jul 15 21:29:54.543577 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:54.543464 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls podName:99581639-234d-413f-be81-2ba54826a4cf nodeName:}" failed. No retries permitted until 2026-07-15 21:29:55.543449092 +0000 UTC m=+37.046884380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls") pod "image-registry-6dcd789698-x8rkr" (UID: "99581639-234d-413f-be81-2ba54826a4cf") : secret "image-registry-tls" not found Jul 15 21:29:54.553690 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:54.553563 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p7x5z"] Jul 15 21:29:54.556900 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:54.556874 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50607b0d_4426_4d4b_97af_4db2e1e305fa.slice/crio-42a8c7bfc143f0ea7d80c64487341849b43f7b6c500867a547f37e139ea63de3 WatchSource:0}: Error finding container 42a8c7bfc143f0ea7d80c64487341849b43f7b6c500867a547f37e139ea63de3: Status 404 returned error can't find the container with id 42a8c7bfc143f0ea7d80c64487341849b43f7b6c500867a547f37e139ea63de3 Jul 15 21:29:55.211443 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:55.211402 2578 generic.go:356] "Generic (PLEG): container finished" podID="dfb01554-4420-4174-a15a-1b67f0dfd799" containerID="d10bd09836972d88dd875d4b367209478ab6ffe12899aa52f8876df621451f51" exitCode=0 Jul 15 21:29:55.212155 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:55.211486 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfpjx" event={"ID":"dfb01554-4420-4174-a15a-1b67f0dfd799","Type":"ContainerDied","Data":"d10bd09836972d88dd875d4b367209478ab6ffe12899aa52f8876df621451f51"} Jul 15 21:29:55.212992 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:55.212963 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p7x5z" event={"ID":"50607b0d-4426-4d4b-97af-4db2e1e305fa","Type":"ContainerStarted","Data":"42a8c7bfc143f0ea7d80c64487341849b43f7b6c500867a547f37e139ea63de3"} Jul 15 21:29:55.553832 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:55.553787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:55.554034 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:55.553844 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:55.554034 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:55.553953 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:29:55.554034 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:55.553981 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcd789698-x8rkr: secret "image-registry-tls" not found Jul 15 21:29:55.554034 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:55.553955 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:29:55.554244 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:55.554053 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls podName:99581639-234d-413f-be81-2ba54826a4cf nodeName:}" failed. No retries permitted until 2026-07-15 21:29:57.554036056 +0000 UTC m=+39.057471337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls") pod "image-registry-6dcd789698-x8rkr" (UID: "99581639-234d-413f-be81-2ba54826a4cf") : secret "image-registry-tls" not found Jul 15 21:29:55.554244 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:55.554070 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls podName:07138502-cc97-44da-9b9f-a0ea92aa4e68 nodeName:}" failed. No retries permitted until 2026-07-15 21:29:57.554063043 +0000 UTC m=+39.057498330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls") pod "dns-default-ml6fl" (UID: "07138502-cc97-44da-9b9f-a0ea92aa4e68") : secret "dns-default-metrics-tls" not found Jul 15 21:29:56.219726 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.219687 2578 generic.go:356] "Generic (PLEG): container finished" podID="dfb01554-4420-4174-a15a-1b67f0dfd799" containerID="f4572b9135c195fe0e9a90cc9cac511e95bf0a0a4425aa84997215a946fcb0f9" exitCode=0 Jul 15 21:29:56.220214 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.219761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfpjx" event={"ID":"dfb01554-4420-4174-a15a-1b67f0dfd799","Type":"ContainerDied","Data":"f4572b9135c195fe0e9a90cc9cac511e95bf0a0a4425aa84997215a946fcb0f9"} Jul 15 21:29:56.255235 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.255197 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:56.324057 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.324020 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:56.326682 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.326660 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-npn72"] Jul 15 21:29:56.361551 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.361515 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-npn72"] Jul 15 21:29:56.361740 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.361666 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:29:56.364374 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.364340 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 15 21:29:56.364374 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.364370 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zmtw2\"" Jul 15 21:29:56.364554 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.364383 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 15 21:29:56.427606 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.427568 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:56.440510 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.440480 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5rrtl"] Jul 15 21:29:56.462451 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.462400 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-npn72\" (UID: \"0ae8a9c8-1f23-41d7-8983-7133759435e6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:29:56.462613 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.462512 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0ae8a9c8-1f23-41d7-8983-7133759435e6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-npn72\" (UID: \"0ae8a9c8-1f23-41d7-8983-7133759435e6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:29:56.463416 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.463391 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5rrtl"] Jul 15 21:29:56.463537 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.463530 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.466395 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.466361 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 21:29:56.466589 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.466557 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 21:29:56.466718 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.466661 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w8f6x\"" Jul 15 21:29:56.466782 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.466757 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 21:29:56.467242 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.467228 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 21:29:56.563965 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.563929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.564158 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.563979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-crio-socket\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.564158 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.564041 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.564158 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.564127 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-npn72\" (UID: \"0ae8a9c8-1f23-41d7-8983-7133759435e6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:29:56.564158 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.564158 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-data-volume\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.564364 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.564190 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0ae8a9c8-1f23-41d7-8983-7133759435e6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-npn72\" (UID: \"0ae8a9c8-1f23-41d7-8983-7133759435e6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:29:56.564364 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:56.564320 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 21:29:56.564364 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.564341 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fwpp\" (UniqueName: \"kubernetes.io/projected/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-api-access-8fwpp\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.564498 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:56.564399 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert podName:0ae8a9c8-1f23-41d7-8983-7133759435e6 nodeName:}" failed. No retries permitted until 2026-07-15 21:29:57.064377878 +0000 UTC m=+38.567813152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-npn72" (UID: "0ae8a9c8-1f23-41d7-8983-7133759435e6") : secret "networking-console-plugin-cert" not found Jul 15 21:29:56.564944 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.564925 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0ae8a9c8-1f23-41d7-8983-7133759435e6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-npn72\" (UID: \"0ae8a9c8-1f23-41d7-8983-7133759435e6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:29:56.608862 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.608827 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v"] Jul 15 21:29:56.635362 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.635331 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v"] Jul 15 21:29:56.635520 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.635503 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v" Jul 15 21:29:56.639106 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.639071 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 15 21:29:56.639286 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.639071 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-7nz2w\"" Jul 15 21:29:56.639658 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.639640 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 15 21:29:56.665209 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.665172 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fwpp\" (UniqueName: \"kubernetes.io/projected/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-api-access-8fwpp\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.665370 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.665241 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.665370 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.665288 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-crio-socket\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.665370 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.665317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.665370 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.665350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmj7z\" (UniqueName: \"kubernetes.io/projected/c27275b5-8cfc-4080-b948-0faf61ae132c-kube-api-access-gmj7z\") pod \"migrator-7f59b54f6c-4rt8v\" (UID: \"c27275b5-8cfc-4080-b948-0faf61ae132c\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v" Jul 15 21:29:56.665556 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:56.665378 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 21:29:56.665556 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.665422 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-data-volume\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.665556 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:56.665448 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm podName:0cfdbf87-675b-4f5d-bccf-b57a4f27eade nodeName:}" failed. No retries permitted until 2026-07-15 21:29:57.165428099 +0000 UTC m=+38.668863373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5rrtl" (UID: "0cfdbf87-675b-4f5d-bccf-b57a4f27eade") : configmap "kube-rbac-proxy" not found Jul 15 21:29:56.665676 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:56.665594 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 21:29:56.665676 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:56.665639 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls podName:0cfdbf87-675b-4f5d-bccf-b57a4f27eade nodeName:}" failed. No retries permitted until 2026-07-15 21:29:57.165622087 +0000 UTC m=+38.669057364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5rrtl" (UID: "0cfdbf87-675b-4f5d-bccf-b57a4f27eade") : secret "insights-runtime-extractor-tls" not found Jul 15 21:29:56.665765 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.665712 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-data-volume\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.665810 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.665790 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-crio-socket\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.675611 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.675581 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fwpp\" (UniqueName: \"kubernetes.io/projected/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-api-access-8fwpp\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:56.766751 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.766663 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmj7z\" (UniqueName: \"kubernetes.io/projected/c27275b5-8cfc-4080-b948-0faf61ae132c-kube-api-access-gmj7z\") pod \"migrator-7f59b54f6c-4rt8v\" (UID: \"c27275b5-8cfc-4080-b948-0faf61ae132c\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v" Jul 15 21:29:56.775307 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.775281 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmj7z\" (UniqueName: \"kubernetes.io/projected/c27275b5-8cfc-4080-b948-0faf61ae132c-kube-api-access-gmj7z\") pod \"migrator-7f59b54f6c-4rt8v\" (UID: \"c27275b5-8cfc-4080-b948-0faf61ae132c\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v" Jul 15 21:29:56.873393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.873358 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:56.945343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.945297 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v" Jul 15 21:29:56.975058 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:56.974881 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:57.026610 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.026534 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:57.068777 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.068751 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-npn72\" (UID: \"0ae8a9c8-1f23-41d7-8983-7133759435e6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:29:57.068879 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:57.068860 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 21:29:57.068934 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:57.068915 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert podName:0ae8a9c8-1f23-41d7-8983-7133759435e6 nodeName:}" failed. No retries permitted until 2026-07-15 21:29:58.068901647 +0000 UTC m=+39.572336921 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-npn72" (UID: "0ae8a9c8-1f23-41d7-8983-7133759435e6") : secret "networking-console-plugin-cert" not found Jul 15 21:29:57.074622 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.074417 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v"] Jul 15 21:29:57.077938 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:57.077919 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc27275b5_8cfc_4080_b948_0faf61ae132c.slice/crio-deb6eb24df63a52b853644f5c468acfc9c5fc3b2eec0efe9d6635d7b0257d915 WatchSource:0}: Error finding container deb6eb24df63a52b853644f5c468acfc9c5fc3b2eec0efe9d6635d7b0257d915: Status 404 returned error can't find the container with id deb6eb24df63a52b853644f5c468acfc9c5fc3b2eec0efe9d6635d7b0257d915 Jul 15 21:29:57.170405 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.170212 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:57.170405 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.170282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:57.170646 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:57.170460 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 21:29:57.170646 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:57.170526 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls podName:0cfdbf87-675b-4f5d-bccf-b57a4f27eade nodeName:}" failed. No retries permitted until 2026-07-15 21:29:58.170505817 +0000 UTC m=+39.673941091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5rrtl" (UID: "0cfdbf87-675b-4f5d-bccf-b57a4f27eade") : secret "insights-runtime-extractor-tls" not found Jul 15 21:29:57.170646 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:57.170563 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 21:29:57.170646 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:57.170626 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm podName:0cfdbf87-675b-4f5d-bccf-b57a4f27eade nodeName:}" failed. No retries permitted until 2026-07-15 21:29:58.170607349 +0000 UTC m=+39.674042650 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5rrtl" (UID: "0cfdbf87-675b-4f5d-bccf-b57a4f27eade") : configmap "kube-rbac-proxy" not found Jul 15 21:29:57.227919 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.227882 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfpjx" event={"ID":"dfb01554-4420-4174-a15a-1b67f0dfd799","Type":"ContainerStarted","Data":"558337a5792185698b36cda810b3c14ab14b610dae2d5ed1b049e9c2a3141f07"} Jul 15 21:29:57.229515 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.229486 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p7x5z" event={"ID":"50607b0d-4426-4d4b-97af-4db2e1e305fa","Type":"ContainerStarted","Data":"cb0d5f1cb925a05fbd1dfa1ee1d726278bd15ecb42daa893f2ffc895d8cc1c65"} Jul 15 21:29:57.230606 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.230585 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v" event={"ID":"c27275b5-8cfc-4080-b948-0faf61ae132c","Type":"ContainerStarted","Data":"deb6eb24df63a52b853644f5c468acfc9c5fc3b2eec0efe9d6635d7b0257d915"} Jul 15 21:29:57.249593 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.249532 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rfpjx" podStartSLOduration=10.933653094 podStartE2EDuration="38.249508874s" totalImagesPullingTime="27.31585578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:57.247671349 +0000 UTC m=+38.751106666" watchObservedRunningTime="2026-07-15 21:29:57.249508874 +0000 UTC m=+38.752944163" Jul 15 21:29:57.262887 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.262837 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p7x5z" podStartSLOduration=1.780721738 podStartE2EDuration="4.262821345s" totalImagesPullingTime="2.482099607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:29:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:57.262373402 +0000 UTC m=+38.765808724" watchObservedRunningTime="2026-07-15 21:29:57.262821345 +0000 UTC m=+38.766256640" Jul 15 21:29:57.388872 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.388841 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:57.438875 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.438842 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:57.573789 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.573755 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:29:57.573963 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.573851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:29:57.573963 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:57.573907 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:29:57.573963 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:57.573925 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcd789698-x8rkr: secret "image-registry-tls" not found Jul 15 21:29:57.573963 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:57.573939 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:29:57.574166 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:57.573985 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls podName:07138502-cc97-44da-9b9f-a0ea92aa4e68 nodeName:}" failed. No retries permitted until 2026-07-15 21:30:01.573972698 +0000 UTC m=+43.077407971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls") pod "dns-default-ml6fl" (UID: "07138502-cc97-44da-9b9f-a0ea92aa4e68") : secret "dns-default-metrics-tls" not found Jul 15 21:29:57.574166 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:57.573998 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls podName:99581639-234d-413f-be81-2ba54826a4cf nodeName:}" failed. No retries permitted until 2026-07-15 21:30:01.573992777 +0000 UTC m=+43.077428050 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls") pod "image-registry-6dcd789698-x8rkr" (UID: "99581639-234d-413f-be81-2ba54826a4cf") : secret "image-registry-tls" not found Jul 15 21:29:57.785790 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.785718 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nbzd8_4d05dda3-b68c-4486-9369-a0615903e606/dns-node-resolver/0.log" Jul 15 21:29:57.793168 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.793129 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:57.843817 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:57.843773 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:58.077791 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.077756 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-npn72\" (UID: \"0ae8a9c8-1f23-41d7-8983-7133759435e6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:29:58.077984 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:58.077920 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 21:29:58.077984 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:58.077983 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert podName:0ae8a9c8-1f23-41d7-8983-7133759435e6 nodeName:}" failed. No retries permitted until 2026-07-15 21:30:00.077967376 +0000 UTC m=+41.581402653 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-npn72" (UID: "0ae8a9c8-1f23-41d7-8983-7133759435e6") : secret "networking-console-plugin-cert" not found Jul 15 21:29:58.178669 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.178629 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:58.178836 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.178684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:29:58.178836 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:58.178785 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 21:29:58.178836 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:58.178812 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 21:29:58.178962 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:58.178868 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm podName:0cfdbf87-675b-4f5d-bccf-b57a4f27eade nodeName:}" failed. No retries permitted until 2026-07-15 21:30:00.178846638 +0000 UTC m=+41.682281917 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5rrtl" (UID: "0cfdbf87-675b-4f5d-bccf-b57a4f27eade") : configmap "kube-rbac-proxy" not found Jul 15 21:29:58.178962 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:29:58.178887 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls podName:0cfdbf87-675b-4f5d-bccf-b57a4f27eade nodeName:}" failed. No retries permitted until 2026-07-15 21:30:00.178877116 +0000 UTC m=+41.682312393 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5rrtl" (UID: "0cfdbf87-675b-4f5d-bccf-b57a4f27eade") : secret "insights-runtime-extractor-tls" not found Jul 15 21:29:58.552916 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.552884 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-fnqmq"] Jul 15 21:29:58.573164 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.573133 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-fnqmq"] Jul 15 21:29:58.573330 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.573269 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" Jul 15 21:29:58.576038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.576016 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 15 21:29:58.576171 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.576156 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 15 21:29:58.577025 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.577009 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 15 21:29:58.577138 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.577080 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-8gdqv\"" Jul 15 21:29:58.577138 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.577127 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 15 21:29:58.589380 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.589358 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-822j5_f191e06a-8191-4762-8e60-52d29e065cb2/node-ca/0.log" Jul 15 21:29:58.589665 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.589647 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:58.638705 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.638674 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:58.682976 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.682943 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffjtf\" (UniqueName: \"kubernetes.io/projected/79f7b4bc-503e-452b-9273-4cc264ba0ad7-kube-api-access-ffjtf\") pod \"service-ca-79f68b9f4f-fnqmq\" (UID: \"79f7b4bc-503e-452b-9273-4cc264ba0ad7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" Jul 15 21:29:58.683150 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.682985 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/79f7b4bc-503e-452b-9273-4cc264ba0ad7-signing-cabundle\") pod \"service-ca-79f68b9f4f-fnqmq\" (UID: \"79f7b4bc-503e-452b-9273-4cc264ba0ad7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" Jul 15 21:29:58.683150 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.683053 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/79f7b4bc-503e-452b-9273-4cc264ba0ad7-signing-key\") pod \"service-ca-79f68b9f4f-fnqmq\" (UID: \"79f7b4bc-503e-452b-9273-4cc264ba0ad7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" Jul 15 21:29:58.783957 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.783923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffjtf\" (UniqueName: \"kubernetes.io/projected/79f7b4bc-503e-452b-9273-4cc264ba0ad7-kube-api-access-ffjtf\") pod \"service-ca-79f68b9f4f-fnqmq\" (UID: \"79f7b4bc-503e-452b-9273-4cc264ba0ad7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" Jul 15 21:29:58.783957 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.783964 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/79f7b4bc-503e-452b-9273-4cc264ba0ad7-signing-cabundle\") pod \"service-ca-79f68b9f4f-fnqmq\" (UID: \"79f7b4bc-503e-452b-9273-4cc264ba0ad7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" Jul 15 21:29:58.784233 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.784159 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/79f7b4bc-503e-452b-9273-4cc264ba0ad7-signing-key\") pod \"service-ca-79f68b9f4f-fnqmq\" (UID: \"79f7b4bc-503e-452b-9273-4cc264ba0ad7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" Jul 15 21:29:58.784692 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.784673 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/79f7b4bc-503e-452b-9273-4cc264ba0ad7-signing-cabundle\") pod \"service-ca-79f68b9f4f-fnqmq\" (UID: \"79f7b4bc-503e-452b-9273-4cc264ba0ad7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" Jul 15 21:29:58.787502 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.787487 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/79f7b4bc-503e-452b-9273-4cc264ba0ad7-signing-key\") pod \"service-ca-79f68b9f4f-fnqmq\" (UID: \"79f7b4bc-503e-452b-9273-4cc264ba0ad7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" Jul 15 21:29:58.791827 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.791798 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffjtf\" (UniqueName: \"kubernetes.io/projected/79f7b4bc-503e-452b-9273-4cc264ba0ad7-kube-api-access-ffjtf\") pod \"service-ca-79f68b9f4f-fnqmq\" (UID: \"79f7b4bc-503e-452b-9273-4cc264ba0ad7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" Jul 15 21:29:58.882775 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:58.882743 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" Jul 15 21:29:59.060562 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:59.060258 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-fnqmq"] Jul 15 21:29:59.063544 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:29:59.063510 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f7b4bc_503e_452b_9273_4cc264ba0ad7.slice/crio-1b7dfd3662c216fc158aa7fe4212fffc353dce7c5620520457b13bf29107f680 WatchSource:0}: Error finding container 1b7dfd3662c216fc158aa7fe4212fffc353dce7c5620520457b13bf29107f680: Status 404 returned error can't find the container with id 1b7dfd3662c216fc158aa7fe4212fffc353dce7c5620520457b13bf29107f680 Jul 15 21:29:59.171244 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:59.171208 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:59.237918 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:59.237881 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v" event={"ID":"c27275b5-8cfc-4080-b948-0faf61ae132c","Type":"ContainerStarted","Data":"5017db56ca334bef6ecb7ae2b5ec06203401e2c45aca72d69f73c253f0e44e42"} Jul 15 21:29:59.238138 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:59.237941 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v" event={"ID":"c27275b5-8cfc-4080-b948-0faf61ae132c","Type":"ContainerStarted","Data":"ad6d8d55680b376056e5e66b977287d41b0aa0ea3fbe0455b1618b1cbfa5300e"} Jul 15 21:29:59.238951 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:59.238923 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" event={"ID":"79f7b4bc-503e-452b-9273-4cc264ba0ad7","Type":"ContainerStarted","Data":"1b7dfd3662c216fc158aa7fe4212fffc353dce7c5620520457b13bf29107f680"} Jul 15 21:29:59.255182 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:59.255133 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4rt8v" podStartSLOduration=1.51601051 podStartE2EDuration="3.255118035s" totalImagesPullingTime="1.739107525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:29:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:59.254754458 +0000 UTC m=+40.758189752" watchObservedRunningTime="2026-07-15 21:29:59.255118035 +0000 UTC m=+40.758553323" Jul 15 21:29:59.386128 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:59.386028 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p7x5z_50607b0d-4426-4d4b-97af-4db2e1e305fa/serve-healthcheck-canary/0.log" Jul 15 21:29:59.414818 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:29:59.414790 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:00.097017 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:00.096967 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-npn72\" (UID: \"0ae8a9c8-1f23-41d7-8983-7133759435e6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:30:00.097496 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:00.097162 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 21:30:00.097496 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:00.097227 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert podName:0ae8a9c8-1f23-41d7-8983-7133759435e6 nodeName:}" failed. No retries permitted until 2026-07-15 21:30:04.097212373 +0000 UTC m=+45.600647647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-npn72" (UID: "0ae8a9c8-1f23-41d7-8983-7133759435e6") : secret "networking-console-plugin-cert" not found Jul 15 21:30:00.198366 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:00.198334 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:30:00.198550 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:00.198436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:30:00.198550 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:00.198498 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 21:30:00.198550 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:00.198500 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 21:30:00.198550 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:00.198547 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm podName:0cfdbf87-675b-4f5d-bccf-b57a4f27eade nodeName:}" failed. No retries permitted until 2026-07-15 21:30:04.198532676 +0000 UTC m=+45.701967950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5rrtl" (UID: "0cfdbf87-675b-4f5d-bccf-b57a4f27eade") : configmap "kube-rbac-proxy" not found Jul 15 21:30:00.198702 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:00.198560 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls podName:0cfdbf87-675b-4f5d-bccf-b57a4f27eade nodeName:}" failed. No retries permitted until 2026-07-15 21:30:04.198554064 +0000 UTC m=+45.701989337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5rrtl" (UID: "0cfdbf87-675b-4f5d-bccf-b57a4f27eade") : secret "insights-runtime-extractor-tls" not found Jul 15 21:30:01.609177 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:01.609148 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:30:01.609495 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:01.609280 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:30:01.609495 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:01.609307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:30:01.609495 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:01.609334 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls podName:07138502-cc97-44da-9b9f-a0ea92aa4e68 nodeName:}" failed. No retries permitted until 2026-07-15 21:30:09.609319578 +0000 UTC m=+51.112754851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls") pod "dns-default-ml6fl" (UID: "07138502-cc97-44da-9b9f-a0ea92aa4e68") : secret "dns-default-metrics-tls" not found Jul 15 21:30:01.609495 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:01.609410 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:30:01.609495 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:01.609424 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcd789698-x8rkr: secret "image-registry-tls" not found Jul 15 21:30:01.609495 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:01.609471 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls podName:99581639-234d-413f-be81-2ba54826a4cf nodeName:}" failed. No retries permitted until 2026-07-15 21:30:09.609456247 +0000 UTC m=+51.112891542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls") pod "image-registry-6dcd789698-x8rkr" (UID: "99581639-234d-413f-be81-2ba54826a4cf") : secret "image-registry-tls" not found Jul 15 21:30:01.945726 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:01.945641 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:02.247451 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:02.247359 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" event={"ID":"79f7b4bc-503e-452b-9273-4cc264ba0ad7","Type":"ContainerStarted","Data":"253eca65741f67fd36b49476daee10f05c6dd5138cf25935b54bf783cab76c27"} Jul 15 21:30:02.267462 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:02.267403 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-fnqmq" podStartSLOduration=1.747984156 podStartE2EDuration="4.267384566s" totalImagesPullingTime="2.51940041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:02.262302717 +0000 UTC m=+43.765738010" watchObservedRunningTime="2026-07-15 21:30:02.267384566 +0000 UTC m=+43.770819860" Jul 15 21:30:02.750566 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:02.750535 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:03.351617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:03.351577 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:03.453484 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:03.453431 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:03.554623 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:03.554565 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:03.654350 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:03.654234 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:03.711182 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:03.711145 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:04.133464 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:04.133411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-npn72\" (UID: \"0ae8a9c8-1f23-41d7-8983-7133759435e6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:30:04.133867 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:04.133573 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 21:30:04.133867 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:04.133643 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert podName:0ae8a9c8-1f23-41d7-8983-7133759435e6 nodeName:}" failed. No retries permitted until 2026-07-15 21:30:12.13362695 +0000 UTC m=+53.637062223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-npn72" (UID: "0ae8a9c8-1f23-41d7-8983-7133759435e6") : secret "networking-console-plugin-cert" not found Jul 15 21:30:04.234371 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:04.234319 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:30:04.234568 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:04.234398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:30:04.234568 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:04.234556 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 21:30:04.234686 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:04.234641 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls podName:0cfdbf87-675b-4f5d-bccf-b57a4f27eade nodeName:}" failed. No retries permitted until 2026-07-15 21:30:12.234619078 +0000 UTC m=+53.738054359 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5rrtl" (UID: "0cfdbf87-675b-4f5d-bccf-b57a4f27eade") : secret "insights-runtime-extractor-tls" not found Jul 15 21:30:04.234964 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:04.234940 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:30:04.245468 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:04.245438 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:04.647552 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:04.647513 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:05.248961 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:05.248931 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:05.847823 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:05.847791 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:06.044430 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:06.044394 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:06.094656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:06.094615 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:06.246642 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:06.246554 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:06.608699 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:06.608661 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:06.709405 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:06.709370 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:07.044666 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:07.044634 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:07.146330 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:07.146286 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:07.244790 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:07.244758 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:07.843981 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:07.843948 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:07.918473 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:07.918439 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:08.049504 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:08.049472 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:08.448248 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:08.448216 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:08.693409 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:08.693377 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:08.747721 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:08.747633 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:09.676790 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:09.676749 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:30:09.677303 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:09.676870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:30:09.680114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:09.680074 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07138502-cc97-44da-9b9f-a0ea92aa4e68-metrics-tls\") pod \"dns-default-ml6fl\" (UID: \"07138502-cc97-44da-9b9f-a0ea92aa4e68\") " pod="openshift-dns/dns-default-ml6fl" Jul 15 21:30:09.680232 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:09.680112 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls\") pod \"image-registry-6dcd789698-x8rkr\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:30:09.771245 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:09.771211 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:30:09.783800 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:09.783772 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ml6fl" Jul 15 21:30:09.815158 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:09.815126 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:09.913856 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:09.913825 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:09.922586 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:09.922558 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dcd789698-x8rkr"] Jul 15 21:30:09.926201 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:09.926175 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99581639_234d_413f_be81_2ba54826a4cf.slice/crio-1a1a8af3e58f1b90df2fa27672530a4d6d17bf773aff700266d8480e72d5e02f WatchSource:0}: Error finding container 1a1a8af3e58f1b90df2fa27672530a4d6d17bf773aff700266d8480e72d5e02f: Status 404 returned error can't find the container with id 1a1a8af3e58f1b90df2fa27672530a4d6d17bf773aff700266d8480e72d5e02f Jul 15 21:30:09.934647 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:09.934622 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ml6fl"] Jul 15 21:30:09.945650 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:09.945623 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07138502_cc97_44da_9b9f_a0ea92aa4e68.slice/crio-f85683d92f340de7e97c1a5b0899ebf4767f07cc44ac8f365bf1382a2b85912b WatchSource:0}: Error finding container f85683d92f340de7e97c1a5b0899ebf4767f07cc44ac8f365bf1382a2b85912b: Status 404 returned error can't find the container with id f85683d92f340de7e97c1a5b0899ebf4767f07cc44ac8f365bf1382a2b85912b Jul 15 21:30:10.070734 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:10.070693 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:10.259395 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:10.259309 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:10.271333 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:10.271293 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ml6fl" event={"ID":"07138502-cc97-44da-9b9f-a0ea92aa4e68","Type":"ContainerStarted","Data":"f85683d92f340de7e97c1a5b0899ebf4767f07cc44ac8f365bf1382a2b85912b"} Jul 15 21:30:10.273001 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:10.272967 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" event={"ID":"99581639-234d-413f-be81-2ba54826a4cf","Type":"ContainerStarted","Data":"8a4698749be7ede2efad5bc9f1e2685efc4d2e44a707ce05a8f5c604ee6cd429"} Jul 15 21:30:10.273138 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:10.273025 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" event={"ID":"99581639-234d-413f-be81-2ba54826a4cf","Type":"ContainerStarted","Data":"1a1a8af3e58f1b90df2fa27672530a4d6d17bf773aff700266d8480e72d5e02f"} Jul 15 21:30:10.273138 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:10.273051 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:30:10.644618 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:10.644580 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:12.003701 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.003602 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:12.105195 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.105083 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:12.195855 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.195811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-npn72\" (UID: \"0ae8a9c8-1f23-41d7-8983-7133759435e6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:30:12.198140 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.198116 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ae8a9c8-1f23-41d7-8983-7133759435e6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-npn72\" (UID: \"0ae8a9c8-1f23-41d7-8983-7133759435e6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:30:12.254121 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.254000 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:12.271799 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.271770 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" Jul 15 21:30:12.296624 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.296593 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:30:12.298772 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.298728 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cfdbf87-675b-4f5d-bccf-b57a4f27eade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rrtl\" (UID: \"0cfdbf87-675b-4f5d-bccf-b57a4f27eade\") " pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:30:12.375205 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.374840 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5rrtl" Jul 15 21:30:12.421459 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.421389 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" podStartSLOduration=44.42136577 podStartE2EDuration="44.42136577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:10.291928764 +0000 UTC m=+51.795364059" watchObservedRunningTime="2026-07-15 21:30:12.42136577 +0000 UTC m=+53.924801062" Jul 15 21:30:12.421646 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.421622 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-npn72"] Jul 15 21:30:12.430215 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:12.429733 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae8a9c8_1f23_41d7_8983_7133759435e6.slice/crio-1acc1c963fd9558b0200d7874b79981450f3138ba48340431f9132276ec1a0d5 WatchSource:0}: Error finding container 1acc1c963fd9558b0200d7874b79981450f3138ba48340431f9132276ec1a0d5: Status 404 returned error can't find the container with id 1acc1c963fd9558b0200d7874b79981450f3138ba48340431f9132276ec1a0d5 Jul 15 21:30:12.515460 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:12.515388 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5rrtl"] Jul 15 21:30:12.518519 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:12.518494 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cfdbf87_675b_4f5d_bccf_b57a4f27eade.slice/crio-6b25e4a26bfa047821e161295c8c7b1cd4a968de109e7e568852f4c8d6200480 WatchSource:0}: Error finding container 6b25e4a26bfa047821e161295c8c7b1cd4a968de109e7e568852f4c8d6200480: Status 404 returned error can't find the container with id 6b25e4a26bfa047821e161295c8c7b1cd4a968de109e7e568852f4c8d6200480 Jul 15 21:30:13.141441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:13.141383 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:13.242620 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:13.242584 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:13.287837 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:13.287792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ml6fl" event={"ID":"07138502-cc97-44da-9b9f-a0ea92aa4e68","Type":"ContainerStarted","Data":"33dec956a95a341ce6a2998748a1c8fb66794f8b3b429247c62bf6e6a06abcf0"} Jul 15 21:30:13.288028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:13.287869 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ml6fl" event={"ID":"07138502-cc97-44da-9b9f-a0ea92aa4e68","Type":"ContainerStarted","Data":"028c3223da45f2c54ee6ea68c3fb66972ef7901e5cf61ee9de78cb9a65fb9bc4"} Jul 15 21:30:13.288126 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:13.288100 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ml6fl" Jul 15 21:30:13.289444 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:13.289414 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5rrtl" event={"ID":"0cfdbf87-675b-4f5d-bccf-b57a4f27eade","Type":"ContainerStarted","Data":"6b25e4a26bfa047821e161295c8c7b1cd4a968de109e7e568852f4c8d6200480"} Jul 15 21:30:13.290677 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:13.290655 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" event={"ID":"0ae8a9c8-1f23-41d7-8983-7133759435e6","Type":"ContainerStarted","Data":"1acc1c963fd9558b0200d7874b79981450f3138ba48340431f9132276ec1a0d5"} Jul 15 21:30:13.304461 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:13.304417 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ml6fl" podStartSLOduration=18.038219965 podStartE2EDuration="20.304403741s" totalImagesPullingTime="2.266183776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:29:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:13.303176445 +0000 UTC m=+54.806611741" watchObservedRunningTime="2026-07-15 21:30:13.304403741 +0000 UTC m=+54.807839014" Jul 15 21:30:13.650870 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:13.650829 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:13.846359 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:13.846319 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:14.050926 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:14.050893 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:14.296339 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:14.296294 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" event={"ID":"0ae8a9c8-1f23-41d7-8983-7133759435e6","Type":"ContainerStarted","Data":"85c1e14e3daa7bcade8040f518d9223391eac13591dd4362c495835e9db6da2e"} Jul 15 21:30:14.312531 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:14.312420 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-npn72" podStartSLOduration=17.073889856 podStartE2EDuration="18.312398264s" totalImagesPullingTime="1.238508408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:29:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:14.311423414 +0000 UTC m=+55.814858709" watchObservedRunningTime="2026-07-15 21:30:14.312398264 +0000 UTC m=+55.815833561" Jul 15 21:30:19.317891 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.317794 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5rrtl" event={"ID":"0cfdbf87-675b-4f5d-bccf-b57a4f27eade","Type":"ContainerStarted","Data":"6e789c5cf4c541a82bbc774cd7cabb5e5c0eb3ec98c666d2fa74a521724b1cac"} Jul 15 21:30:19.554668 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.554627 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8"] Jul 15 21:30:19.557800 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.557773 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7"] Jul 15 21:30:19.557966 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.557939 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8" Jul 15 21:30:19.560871 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.560846 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:19.561082 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.561064 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 15 21:30:19.562312 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.562290 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 15 21:30:19.562413 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.562382 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 15 21:30:19.562519 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.562503 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-j564j\"" Jul 15 21:30:19.562634 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.562618 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 15 21:30:19.563751 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.563730 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 15 21:30:19.567738 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.567400 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dcd789698-x8rkr"] Jul 15 21:30:19.576256 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.576170 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8"] Jul 15 21:30:19.577359 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.577290 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7"] Jul 15 21:30:19.658084 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.658053 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgk8k\" (UniqueName: \"kubernetes.io/projected/44f00014-4151-49ca-840d-8564d5e1a2db-kube-api-access-tgk8k\") pod \"managed-serviceaccount-addon-agent-55f5444dd-5xcj8\" (UID: \"44f00014-4151-49ca-840d-8564d5e1a2db\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8" Jul 15 21:30:19.658276 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.658187 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d8be038-e001-4b5f-bcea-6114282943b2-tmp\") pod \"klusterlet-addon-workmgr-6dfc5675fc-8vpd7\" (UID: \"7d8be038-e001-4b5f-bcea-6114282943b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:19.658276 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.658242 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7g2r\" (UniqueName: \"kubernetes.io/projected/7d8be038-e001-4b5f-bcea-6114282943b2-kube-api-access-g7g2r\") pod \"klusterlet-addon-workmgr-6dfc5675fc-8vpd7\" (UID: \"7d8be038-e001-4b5f-bcea-6114282943b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:19.658390 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.658354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44f00014-4151-49ca-840d-8564d5e1a2db-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55f5444dd-5xcj8\" (UID: \"44f00014-4151-49ca-840d-8564d5e1a2db\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8" Jul 15 21:30:19.658436 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.658385 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7d8be038-e001-4b5f-bcea-6114282943b2-klusterlet-config\") pod \"klusterlet-addon-workmgr-6dfc5675fc-8vpd7\" (UID: \"7d8be038-e001-4b5f-bcea-6114282943b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:19.667270 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.667240 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59d7c4c88f-z5dc7"] Jul 15 21:30:19.670560 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.670533 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.679190 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.678705 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:19.685819 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.685790 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59d7c4c88f-z5dc7"] Jul 15 21:30:19.759151 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759112 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fbc3b38c-86f5-49d4-a0b8-a4616817759b-ca-trust-extracted\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.759381 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759164 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgk8k\" (UniqueName: \"kubernetes.io/projected/44f00014-4151-49ca-840d-8564d5e1a2db-kube-api-access-tgk8k\") pod \"managed-serviceaccount-addon-agent-55f5444dd-5xcj8\" (UID: \"44f00014-4151-49ca-840d-8564d5e1a2db\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8" Jul 15 21:30:19.759381 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759240 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fbc3b38c-86f5-49d4-a0b8-a4616817759b-registry-certificates\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.759381 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvpzl\" (UniqueName: \"kubernetes.io/projected/fbc3b38c-86f5-49d4-a0b8-a4616817759b-kube-api-access-zvpzl\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.759381 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759297 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fbc3b38c-86f5-49d4-a0b8-a4616817759b-trusted-ca\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.759381 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759357 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fbc3b38c-86f5-49d4-a0b8-a4616817759b-bound-sa-token\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.759381 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759379 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fbc3b38c-86f5-49d4-a0b8-a4616817759b-installation-pull-secrets\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.759637 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fbc3b38c-86f5-49d4-a0b8-a4616817759b-registry-tls\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.759637 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d8be038-e001-4b5f-bcea-6114282943b2-tmp\") pod \"klusterlet-addon-workmgr-6dfc5675fc-8vpd7\" (UID: \"7d8be038-e001-4b5f-bcea-6114282943b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:19.759637 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7g2r\" (UniqueName: \"kubernetes.io/projected/7d8be038-e001-4b5f-bcea-6114282943b2-kube-api-access-g7g2r\") pod \"klusterlet-addon-workmgr-6dfc5675fc-8vpd7\" (UID: \"7d8be038-e001-4b5f-bcea-6114282943b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:19.759637 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759526 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44f00014-4151-49ca-840d-8564d5e1a2db-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55f5444dd-5xcj8\" (UID: \"44f00014-4151-49ca-840d-8564d5e1a2db\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8" Jul 15 21:30:19.759637 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7d8be038-e001-4b5f-bcea-6114282943b2-klusterlet-config\") pod \"klusterlet-addon-workmgr-6dfc5675fc-8vpd7\" (UID: \"7d8be038-e001-4b5f-bcea-6114282943b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:19.759637 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759594 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fbc3b38c-86f5-49d4-a0b8-a4616817759b-image-registry-private-configuration\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.759907 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.759871 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d8be038-e001-4b5f-bcea-6114282943b2-tmp\") pod \"klusterlet-addon-workmgr-6dfc5675fc-8vpd7\" (UID: \"7d8be038-e001-4b5f-bcea-6114282943b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:19.762403 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.762370 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7d8be038-e001-4b5f-bcea-6114282943b2-klusterlet-config\") pod \"klusterlet-addon-workmgr-6dfc5675fc-8vpd7\" (UID: \"7d8be038-e001-4b5f-bcea-6114282943b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:19.762635 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.762614 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44f00014-4151-49ca-840d-8564d5e1a2db-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55f5444dd-5xcj8\" (UID: \"44f00014-4151-49ca-840d-8564d5e1a2db\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8" Jul 15 21:30:19.771775 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.771720 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:19.772296 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.772247 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgk8k\" (UniqueName: \"kubernetes.io/projected/44f00014-4151-49ca-840d-8564d5e1a2db-kube-api-access-tgk8k\") pod \"managed-serviceaccount-addon-agent-55f5444dd-5xcj8\" (UID: \"44f00014-4151-49ca-840d-8564d5e1a2db\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8" Jul 15 21:30:19.772847 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.772789 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7g2r\" (UniqueName: \"kubernetes.io/projected/7d8be038-e001-4b5f-bcea-6114282943b2-kube-api-access-g7g2r\") pod \"klusterlet-addon-workmgr-6dfc5675fc-8vpd7\" (UID: \"7d8be038-e001-4b5f-bcea-6114282943b2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:19.860160 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.860029 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fbc3b38c-86f5-49d4-a0b8-a4616817759b-image-registry-private-configuration\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.860160 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.860112 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fbc3b38c-86f5-49d4-a0b8-a4616817759b-ca-trust-extracted\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.860377 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.860165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fbc3b38c-86f5-49d4-a0b8-a4616817759b-registry-certificates\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.860377 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.860190 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvpzl\" (UniqueName: \"kubernetes.io/projected/fbc3b38c-86f5-49d4-a0b8-a4616817759b-kube-api-access-zvpzl\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.860607 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.860230 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fbc3b38c-86f5-49d4-a0b8-a4616817759b-trusted-ca\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.860746 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.860704 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fbc3b38c-86f5-49d4-a0b8-a4616817759b-bound-sa-token\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.860801 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.860753 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fbc3b38c-86f5-49d4-a0b8-a4616817759b-installation-pull-secrets\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.860849 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.860802 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fbc3b38c-86f5-49d4-a0b8-a4616817759b-registry-tls\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.861465 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.861440 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fbc3b38c-86f5-49d4-a0b8-a4616817759b-ca-trust-extracted\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.863686 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.862287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fbc3b38c-86f5-49d4-a0b8-a4616817759b-registry-certificates\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.866344 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.863935 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fbc3b38c-86f5-49d4-a0b8-a4616817759b-trusted-ca\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.866344 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.864113 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fbc3b38c-86f5-49d4-a0b8-a4616817759b-image-registry-private-configuration\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.866344 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.864408 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fbc3b38c-86f5-49d4-a0b8-a4616817759b-installation-pull-secrets\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.868915 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.868885 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fbc3b38c-86f5-49d4-a0b8-a4616817759b-registry-tls\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.871329 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.871296 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:19.872844 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.872817 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fbc3b38c-86f5-49d4-a0b8-a4616817759b-bound-sa-token\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.872958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.872914 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvpzl\" (UniqueName: \"kubernetes.io/projected/fbc3b38c-86f5-49d4-a0b8-a4616817759b-kube-api-access-zvpzl\") pod \"image-registry-59d7c4c88f-z5dc7\" (UID: \"fbc3b38c-86f5-49d4-a0b8-a4616817759b\") " pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:19.879485 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.879256 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8" Jul 15 21:30:19.887729 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.887699 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:19.967453 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.967400 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:19.983396 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:19.983369 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:20.031775 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.031429 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8"] Jul 15 21:30:20.035637 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:20.035605 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44f00014_4151_49ca_840d_8564d5e1a2db.slice/crio-ffc03d3a057761b2f891dc61ca2d498b5516841f25059bac1f6ddcedecb50285 WatchSource:0}: Error finding container ffc03d3a057761b2f891dc61ca2d498b5516841f25059bac1f6ddcedecb50285: Status 404 returned error can't find the container with id ffc03d3a057761b2f891dc61ca2d498b5516841f25059bac1f6ddcedecb50285 Jul 15 21:30:20.052740 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.052627 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7"] Jul 15 21:30:20.056513 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:20.056483 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d8be038_e001_4b5f_bcea_6114282943b2.slice/crio-0132b19ada8a97e2ad9abe70305228d6fb11e3017016c5763af7d85736151883 WatchSource:0}: Error finding container 0132b19ada8a97e2ad9abe70305228d6fb11e3017016c5763af7d85736151883: Status 404 returned error can't find the container with id 0132b19ada8a97e2ad9abe70305228d6fb11e3017016c5763af7d85736151883 Jul 15 21:30:20.126742 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.126654 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59d7c4c88f-z5dc7"] Jul 15 21:30:20.131875 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:20.131845 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc3b38c_86f5_49d4_a0b8_a4616817759b.slice/crio-ea80f048528987ad072c371e82ff99df41c4367004a173ccb4ca8e6b851f59bf WatchSource:0}: Error finding container ea80f048528987ad072c371e82ff99df41c4367004a173ccb4ca8e6b851f59bf: Status 404 returned error can't find the container with id ea80f048528987ad072c371e82ff99df41c4367004a173ccb4ca8e6b851f59bf Jul 15 21:30:20.207099 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.207061 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqhrr" Jul 15 21:30:20.323369 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.323326 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" event={"ID":"fbc3b38c-86f5-49d4-a0b8-a4616817759b","Type":"ContainerStarted","Data":"76449201f3ab38d0db429a5a637583206227f35f1fcbbe6732dbaef42a5f1d60"} Jul 15 21:30:20.323825 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.323384 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" event={"ID":"fbc3b38c-86f5-49d4-a0b8-a4616817759b","Type":"ContainerStarted","Data":"ea80f048528987ad072c371e82ff99df41c4367004a173ccb4ca8e6b851f59bf"} Jul 15 21:30:20.323825 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.323407 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:20.325017 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.324979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" event={"ID":"7d8be038-e001-4b5f-bcea-6114282943b2","Type":"ContainerStarted","Data":"0132b19ada8a97e2ad9abe70305228d6fb11e3017016c5763af7d85736151883"} Jul 15 21:30:20.326274 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.326244 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8" event={"ID":"44f00014-4151-49ca-840d-8564d5e1a2db","Type":"ContainerStarted","Data":"ffc03d3a057761b2f891dc61ca2d498b5516841f25059bac1f6ddcedecb50285"} Jul 15 21:30:20.341578 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.341524 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" podStartSLOduration=1.341509472 podStartE2EDuration="1.341509472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:20.340350597 +0000 UTC m=+61.843785894" watchObservedRunningTime="2026-07-15 21:30:20.341509472 +0000 UTC m=+61.844944745" Jul 15 21:30:20.585783 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.585742 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:20.636417 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.636357 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:20.980108 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:20.979946 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:21.035702 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:21.035663 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:23.299196 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:23.298905 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ml6fl" Jul 15 21:30:23.340351 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:23.340310 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5rrtl" event={"ID":"0cfdbf87-675b-4f5d-bccf-b57a4f27eade","Type":"ContainerStarted","Data":"878b81b6d7ed79513a2297e4826d6802f465dae709607b63adee24d7674c093f"} Jul 15 21:30:23.412165 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:23.412116 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:23.463536 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:23.463346 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:23.868486 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:23.868437 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:23.949949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:23.949915 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:24.039604 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.039568 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:24.095341 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.095248 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:24.162163 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.162071 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:24.216312 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.216220 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:24.861801 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.861746 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:24.910019 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.909980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:30:24.910216 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.910109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7zmr\" (UniqueName: \"kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr\") pod \"network-check-target-t9g8r\" (UID: \"0037cbd3-6cda-4518-a2b0-db580fa02577\") " pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:30:24.910216 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.910160 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:30:24.912965 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.912937 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 21:30:24.913895 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.913873 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 21:30:24.914019 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.913880 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 21:30:24.923627 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.923596 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 21:30:24.923824 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.923791 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50eee8aa-8304-424c-8525-91b270d118ac-metrics-certs\") pod \"network-metrics-daemon-j7s97\" (UID: \"50eee8aa-8304-424c-8525-91b270d118ac\") " pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:30:24.924003 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.923976 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52934511-13c5-4254-bdbd-b3e67a77c80f-original-pull-secret\") pod \"global-pull-secret-syncer-6jqqs\" (UID: \"52934511-13c5-4254-bdbd-b3e67a77c80f\") " pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:30:24.929815 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.929788 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:24.934650 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.934620 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7zmr\" (UniqueName: \"kubernetes.io/projected/0037cbd3-6cda-4518-a2b0-db580fa02577-kube-api-access-d7zmr\") pod \"network-check-target-t9g8r\" (UID: \"0037cbd3-6cda-4518-a2b0-db580fa02577\") " pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:30:24.967105 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.967059 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jqqs" Jul 15 21:30:24.976124 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.976080 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8nwm9\"" Jul 15 21:30:24.980308 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.980286 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-75svw\"" Jul 15 21:30:24.983457 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.983437 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7s97" Jul 15 21:30:24.988688 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:24.988666 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:30:25.260577 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:25.260492 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:25.360309 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:25.360277 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:25.839801 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:25.839755 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:25.940780 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:25.940744 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:26.455345 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:26.455315 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm"] Jul 15 21:30:26.478860 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:26.478823 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm"] Jul 15 21:30:26.479046 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:26.478917 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm" Jul 15 21:30:26.482452 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:26.482436 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 15 21:30:26.482812 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:26.482793 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vjmzb\"" Jul 15 21:30:26.623937 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:26.623902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/086a628f-3fb3-4dce-b92c-b6ba7b203703-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-rjhbm\" (UID: \"086a628f-3fb3-4dce-b92c-b6ba7b203703\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm" Jul 15 21:30:26.724419 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:26.724329 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/086a628f-3fb3-4dce-b92c-b6ba7b203703-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-rjhbm\" (UID: \"086a628f-3fb3-4dce-b92c-b6ba7b203703\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm" Jul 15 21:30:26.727066 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:26.727037 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/086a628f-3fb3-4dce-b92c-b6ba7b203703-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-rjhbm\" (UID: \"086a628f-3fb3-4dce-b92c-b6ba7b203703\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm" Jul 15 21:30:26.746051 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:26.746009 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:26.790870 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:26.790826 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm" Jul 15 21:30:26.846855 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:26.846819 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:27.941127 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:27.941038 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j7s97"] Jul 15 21:30:27.944385 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:27.944342 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50eee8aa_8304_424c_8525_91b270d118ac.slice/crio-eaef4e90cd14e016ee3f27387aa86ed415264281eaee9ac813298f16065bf1c2 WatchSource:0}: Error finding container eaef4e90cd14e016ee3f27387aa86ed415264281eaee9ac813298f16065bf1c2: Status 404 returned error can't find the container with id eaef4e90cd14e016ee3f27387aa86ed415264281eaee9ac813298f16065bf1c2 Jul 15 21:30:27.951125 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:27.951101 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t9g8r"] Jul 15 21:30:28.085209 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.085169 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6jqqs"] Jul 15 21:30:28.088103 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.088057 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm"] Jul 15 21:30:28.088279 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:28.088255 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52934511_13c5_4254_bdbd_b3e67a77c80f.slice/crio-e62bb9f8546aa969771bb8b94194333889425c835a4ae450d0b96e84126398c9 WatchSource:0}: Error finding container e62bb9f8546aa969771bb8b94194333889425c835a4ae450d0b96e84126398c9: Status 404 returned error can't find the container with id e62bb9f8546aa969771bb8b94194333889425c835a4ae450d0b96e84126398c9 Jul 15 21:30:28.091545 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:28.091521 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086a628f_3fb3_4dce_b92c_b6ba7b203703.slice/crio-6bbf48defc154cffb44d1c1ae27c115e0d359157d6ff208a3878976c4c178bd5 WatchSource:0}: Error finding container 6bbf48defc154cffb44d1c1ae27c115e0d359157d6ff208a3878976c4c178bd5: Status 404 returned error can't find the container with id 6bbf48defc154cffb44d1c1ae27c115e0d359157d6ff208a3878976c4c178bd5 Jul 15 21:30:28.359241 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.359194 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8" event={"ID":"44f00014-4151-49ca-840d-8564d5e1a2db","Type":"ContainerStarted","Data":"de6d2bae785e83681e57ab099077cb0df9938cf7cf1c355b0e52b182e477947d"} Jul 15 21:30:28.361613 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.361585 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5rrtl" event={"ID":"0cfdbf87-675b-4f5d-bccf-b57a4f27eade","Type":"ContainerStarted","Data":"1bf4cdb603cbbdd8d35a25d8bd678e20cd66fbb0214cc5498922168fd0f638ab"} Jul 15 21:30:28.363084 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.363055 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t9g8r" event={"ID":"0037cbd3-6cda-4518-a2b0-db580fa02577","Type":"ContainerStarted","Data":"9906208cd9366a81cfe6caca286139771ea2f5f555b08fbd5681eaa1306232de"} Jul 15 21:30:28.364453 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.364408 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm" event={"ID":"086a628f-3fb3-4dce-b92c-b6ba7b203703","Type":"ContainerStarted","Data":"6bbf48defc154cffb44d1c1ae27c115e0d359157d6ff208a3878976c4c178bd5"} Jul 15 21:30:28.365656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.365633 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j7s97" event={"ID":"50eee8aa-8304-424c-8525-91b270d118ac","Type":"ContainerStarted","Data":"eaef4e90cd14e016ee3f27387aa86ed415264281eaee9ac813298f16065bf1c2"} Jul 15 21:30:28.366920 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.366895 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6jqqs" event={"ID":"52934511-13c5-4254-bdbd-b3e67a77c80f","Type":"ContainerStarted","Data":"e62bb9f8546aa969771bb8b94194333889425c835a4ae450d0b96e84126398c9"} Jul 15 21:30:28.368491 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.368457 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" event={"ID":"7d8be038-e001-4b5f-bcea-6114282943b2","Type":"ContainerStarted","Data":"53ad0bbca5fcc46a99559cf57700981318670486f65265879000f79b5345b699"} Jul 15 21:30:28.368749 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.368700 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:28.370519 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.370501 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" Jul 15 21:30:28.390246 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.390184 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55f5444dd-5xcj8" podStartSLOduration=1.729841892 podStartE2EDuration="9.390162426s" totalImagesPullingTime="7.660320534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:28.373677635 +0000 UTC m=+69.877112931" watchObservedRunningTime="2026-07-15 21:30:28.390162426 +0000 UTC m=+69.893597721" Jul 15 21:30:28.390573 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.390544 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dfc5675fc-8vpd7" podStartSLOduration=1.725057488 podStartE2EDuration="9.390533184s" totalImagesPullingTime="7.665475696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:28.388511191 +0000 UTC m=+69.891946485" watchObservedRunningTime="2026-07-15 21:30:28.390533184 +0000 UTC m=+69.893968479" Jul 15 21:30:28.408048 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:28.407984 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5rrtl" podStartSLOduration=17.376018698 podStartE2EDuration="32.407965373s" totalImagesPullingTime="15.031946675s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:28.406763832 +0000 UTC m=+69.910199165" watchObservedRunningTime="2026-07-15 21:30:28.407965373 +0000 UTC m=+69.911400668" Jul 15 21:30:29.376278 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:29.376045 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j7s97" event={"ID":"50eee8aa-8304-424c-8525-91b270d118ac","Type":"ContainerStarted","Data":"16e389317050595ee06d0f7e64f41ba8ac0b8c126db1503a23cbf97b12e429a3"} Jul 15 21:30:29.376278 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:29.376127 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j7s97" event={"ID":"50eee8aa-8304-424c-8525-91b270d118ac","Type":"ContainerStarted","Data":"3687cafc6912e2ccba7f82cfd5a5b192e4f2559c2123b90e115fff20fd107f7a"} Jul 15 21:30:29.400891 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:29.400817 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-j7s97" podStartSLOduration=69.560766773 podStartE2EDuration="1m10.400755223s" totalImagesPullingTime="839.98845ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:29.400013448 +0000 UTC m=+70.903448743" watchObservedRunningTime="2026-07-15 21:30:29.400755223 +0000 UTC m=+70.904190518" Jul 15 21:30:29.520788 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:29.520745 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:29.574425 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:29.574381 2578 patch_prober.go:28] interesting pod/image-registry-6dcd789698-x8rkr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 21:30:29.574585 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:29.574458 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" podUID="99581639-234d-413f-be81-2ba54826a4cf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 21:30:29.621340 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:29.621304 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:30.382073 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:30.382031 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm" event={"ID":"086a628f-3fb3-4dce-b92c-b6ba7b203703","Type":"ContainerStarted","Data":"3932837cd17e5dfd0fa487970503c99a6dcd16cae62763b725ae4e16d59339a4"} Jul 15 21:30:30.382526 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:30.382243 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm" Jul 15 21:30:30.388166 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:30.388138 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm" Jul 15 21:30:30.407889 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:30.407738 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-rjhbm" podStartSLOduration=2.862965002 podStartE2EDuration="4.407717521s" totalImagesPullingTime="1.544752519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:30.407253694 +0000 UTC m=+71.910688989" watchObservedRunningTime="2026-07-15 21:30:30.407717521 +0000 UTC m=+71.911152817" Jul 15 21:30:31.483353 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:31.483312 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:31.547411 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:31.547238 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:31.674179 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:31.674076 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:31.884342 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:31.884311 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-z6vvf"] Jul 15 21:30:31.898808 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:31.898719 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:31.908751 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:31.908719 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-z6vvf"] Jul 15 21:30:31.908909 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:31.908837 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-z6vvf" Jul 15 21:30:31.911602 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:31.911550 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 21:30:31.911602 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:31.911550 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-27rc7\"" Jul 15 21:30:31.911602 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:31.911587 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 21:30:31.986695 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:31.986613 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:32.046788 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.046759 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:32.069902 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.069871 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkdpc\" (UniqueName: \"kubernetes.io/projected/ef049efd-fdaa-48eb-9766-9c4647e076c0-kube-api-access-qkdpc\") pod \"downloads-68c9cbb497-z6vvf\" (UID: \"ef049efd-fdaa-48eb-9766-9c4647e076c0\") " pod="openshift-console/downloads-68c9cbb497-z6vvf" Jul 15 21:30:32.170745 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.170698 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkdpc\" (UniqueName: \"kubernetes.io/projected/ef049efd-fdaa-48eb-9766-9c4647e076c0-kube-api-access-qkdpc\") pod \"downloads-68c9cbb497-z6vvf\" (UID: \"ef049efd-fdaa-48eb-9766-9c4647e076c0\") " pod="openshift-console/downloads-68c9cbb497-z6vvf" Jul 15 21:30:32.180926 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.180889 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkdpc\" (UniqueName: \"kubernetes.io/projected/ef049efd-fdaa-48eb-9766-9c4647e076c0-kube-api-access-qkdpc\") pod \"downloads-68c9cbb497-z6vvf\" (UID: \"ef049efd-fdaa-48eb-9766-9c4647e076c0\") " pod="openshift-console/downloads-68c9cbb497-z6vvf" Jul 15 21:30:32.186009 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.185977 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:32.219858 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.219817 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-z6vvf" Jul 15 21:30:32.391802 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.391717 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t9g8r" event={"ID":"0037cbd3-6cda-4518-a2b0-db580fa02577","Type":"ContainerStarted","Data":"20c1af6f92aede5c2da9ad0daf10b1ad8eabc79f784b6f395958ac60af152ac5"} Jul 15 21:30:32.392021 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.392001 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:30:32.408982 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.408936 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-t9g8r" podStartSLOduration=70.025191017 podStartE2EDuration="1m13.408921867s" totalImagesPullingTime="3.38373085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:32.408873761 +0000 UTC m=+73.912309055" watchObservedRunningTime="2026-07-15 21:30:32.408921867 +0000 UTC m=+73.912357193" Jul 15 21:30:32.417440 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.417406 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:32.469039 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.468995 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:32.568490 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.568458 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:32.669555 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.669519 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:32.972289 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:32.972195 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:33.074224 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:33.074180 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:33.596274 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:33.596239 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:33.618808 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:33.618774 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-z6vvf"] Jul 15 21:30:33.623465 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:33.623436 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef049efd_fdaa_48eb_9766_9c4647e076c0.slice/crio-8055c9e296fcd6c29180f373b31a9520556bb9819c17db25751374bd52e8d705 WatchSource:0}: Error finding container 8055c9e296fcd6c29180f373b31a9520556bb9819c17db25751374bd52e8d705: Status 404 returned error can't find the container with id 8055c9e296fcd6c29180f373b31a9520556bb9819c17db25751374bd52e8d705 Jul 15 21:30:33.696146 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:33.696085 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:33.797497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:33.797459 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:33.899606 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:33.899569 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:34.024606 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:34.024568 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:34.401577 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:34.401541 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6jqqs" event={"ID":"52934511-13c5-4254-bdbd-b3e67a77c80f","Type":"ContainerStarted","Data":"113797694d9856a817e22ecaf5e66002908e96bffdcbd6e17ae9c10743f92e6b"} Jul 15 21:30:34.402760 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:34.402734 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-z6vvf" event={"ID":"ef049efd-fdaa-48eb-9766-9c4647e076c0","Type":"ContainerStarted","Data":"8055c9e296fcd6c29180f373b31a9520556bb9819c17db25751374bd52e8d705"} Jul 15 21:30:34.423870 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:34.423797 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6jqqs" podStartSLOduration=68.989183603 podStartE2EDuration="1m14.42377916s" totalImagesPullingTime="5.434595557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:34.423321776 +0000 UTC m=+75.926757071" watchObservedRunningTime="2026-07-15 21:30:34.42377916 +0000 UTC m=+75.927214457" Jul 15 21:30:34.435183 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:34.435150 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:35.633314 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.633275 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:35.818396 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.818351 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:35.899079 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.898978 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng"] Jul 15 21:30:35.902753 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.902721 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:35.905625 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.905596 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 15 21:30:35.905880 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.905864 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 21:30:35.906054 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.906041 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 21:30:35.906272 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.906256 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wmsj8\"" Jul 15 21:30:35.906609 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.906590 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 15 21:30:35.906789 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.906772 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 21:30:35.913710 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.913679 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sggl4"] Jul 15 21:30:35.917485 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.917459 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-7647x"] Jul 15 21:30:35.919487 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.919462 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:35.921381 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.921220 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng"] Jul 15 21:30:35.921381 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.921344 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:35.923080 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.922845 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 21:30:35.923080 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.922897 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 21:30:35.923080 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.922955 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 21:30:35.923353 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.923337 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g2fkk\"" Jul 15 21:30:35.924261 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.924240 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kckrw\"" Jul 15 21:30:35.924914 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.924888 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 15 21:30:35.925374 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.925337 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 15 21:30:35.925517 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.925479 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 15 21:30:35.937368 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.937347 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:35.938552 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.938531 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-7647x"] Jul 15 21:30:35.998976 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.998936 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e8ad9669-bb51-45d5-82dd-33a17a08a991-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:35.999191 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.998983 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p48wj\" (UniqueName: \"kubernetes.io/projected/e8ad9669-bb51-45d5-82dd-33a17a08a991-kube-api-access-p48wj\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:35.999191 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999010 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:35.999191 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999041 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8ad9669-bb51-45d5-82dd-33a17a08a991-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:35.999191 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:35.999191 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999111 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71b48e14-1112-4ec7-a791-59a3bcb638fd-metrics-client-ca\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:35.999191 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999134 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdwdp\" (UniqueName: \"kubernetes.io/projected/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-api-access-gdwdp\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:35.999191 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999158 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:35.999191 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8ad9669-bb51-45d5-82dd-33a17a08a991-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:35.999578 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-tls\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:35.999578 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999232 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71b48e14-1112-4ec7-a791-59a3bcb638fd-root\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:35.999578 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999471 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-wtmp\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:35.999578 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:35.999578 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999544 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71b48e14-1112-4ec7-a791-59a3bcb638fd-sys\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:35.999820 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999588 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:35.999820 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-textfile\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:35.999820 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999735 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:35.999820 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999785 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.000010 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:35.999833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6cl6\" (UniqueName: \"kubernetes.io/projected/71b48e14-1112-4ec7-a791-59a3bcb638fd-kube-api-access-k6cl6\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.023829 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.023790 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-textfile\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103289 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103352 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6cl6\" (UniqueName: \"kubernetes.io/projected/71b48e14-1112-4ec7-a791-59a3bcb638fd-kube-api-access-k6cl6\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e8ad9669-bb51-45d5-82dd-33a17a08a991-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103477 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p48wj\" (UniqueName: \"kubernetes.io/projected/e8ad9669-bb51-45d5-82dd-33a17a08a991-kube-api-access-p48wj\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103512 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8ad9669-bb51-45d5-82dd-33a17a08a991-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71b48e14-1112-4ec7-a791-59a3bcb638fd-metrics-client-ca\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdwdp\" (UniqueName: \"kubernetes.io/projected/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-api-access-gdwdp\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103683 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103714 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8ad9669-bb51-45d5-82dd-33a17a08a991-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103749 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-tls\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71b48e14-1112-4ec7-a791-59a3bcb638fd-root\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.109114 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-wtmp\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103843 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71b48e14-1112-4ec7-a791-59a3bcb638fd-sys\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.103907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:36.104649 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:36.104732 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-tls podName:71b48e14-1112-4ec7-a791-59a3bcb638fd nodeName:}" failed. No retries permitted until 2026-07-15 21:30:36.6047089 +0000 UTC m=+78.108144177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-tls") pod "node-exporter-sggl4" (UID: "71b48e14-1112-4ec7-a791-59a3bcb638fd") : secret "node-exporter-tls" not found Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.105567 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71b48e14-1112-4ec7-a791-59a3bcb638fd-metrics-client-ca\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:36.105912 2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:36.105965 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-tls podName:1c473fde-ec1e-436a-acf6-c3cf25bd25b7 nodeName:}" failed. No retries permitted until 2026-07-15 21:30:36.605949969 +0000 UTC m=+78.109385248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-7647x" (UID: "1c473fde-ec1e-436a-acf6-c3cf25bd25b7") : secret "kube-state-metrics-tls" not found Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.106267 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-wtmp\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.106333 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71b48e14-1112-4ec7-a791-59a3bcb638fd-root\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.106975 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.107511 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-textfile\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.108163 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.109036 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8ad9669-bb51-45d5-82dd-33a17a08a991-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.109442 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71b48e14-1112-4ec7-a791-59a3bcb638fd-sys\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.110112 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:36.109732 2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 15 21:30:36.110827 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:36.109784 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8ad9669-bb51-45d5-82dd-33a17a08a991-openshift-state-metrics-tls podName:e8ad9669-bb51-45d5-82dd-33a17a08a991 nodeName:}" failed. No retries permitted until 2026-07-15 21:30:36.609762194 +0000 UTC m=+78.113197473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e8ad9669-bb51-45d5-82dd-33a17a08a991-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-ck4ng" (UID: "e8ad9669-bb51-45d5-82dd-33a17a08a991") : secret "openshift-state-metrics-tls" not found Jul 15 21:30:36.110827 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.110071 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.111070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.111049 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.113579 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.113521 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.116510 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.116487 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.116937 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.116912 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6cl6\" (UniqueName: \"kubernetes.io/projected/71b48e14-1112-4ec7-a791-59a3bcb638fd-kube-api-access-k6cl6\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.117488 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.117081 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdwdp\" (UniqueName: \"kubernetes.io/projected/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-api-access-gdwdp\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.119745 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.119691 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p48wj\" (UniqueName: \"kubernetes.io/projected/e8ad9669-bb51-45d5-82dd-33a17a08a991-kube-api-access-p48wj\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:36.120255 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.120217 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8ad9669-bb51-45d5-82dd-33a17a08a991-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:36.172204 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.172022 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:36.273252 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.273075 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:36.373534 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.373391 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:36.439504 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.439384 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:36.608512 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.608480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.608684 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.608522 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-tls\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.611222 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.611201 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71b48e14-1112-4ec7-a791-59a3bcb638fd-node-exporter-tls\") pod \"node-exporter-sggl4\" (UID: \"71b48e14-1112-4ec7-a791-59a3bcb638fd\") " pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.611509 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.611487 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c473fde-ec1e-436a-acf6-c3cf25bd25b7-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-7647x\" (UID: \"1c473fde-ec1e-436a-acf6-c3cf25bd25b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.673792 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.673754 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:36.709893 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.709811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e8ad9669-bb51-45d5-82dd-33a17a08a991-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:36.712746 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.712711 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e8ad9669-bb51-45d5-82dd-33a17a08a991-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-ck4ng\" (UID: \"e8ad9669-bb51-45d5-82dd-33a17a08a991\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:36.774645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.774609 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:36.822422 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.822384 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" Jul 15 21:30:36.842206 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.842177 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sggl4" Jul 15 21:30:36.847761 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.847729 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" Jul 15 21:30:36.858038 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:36.858007 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b48e14_1112_4ec7_a791_59a3bcb638fd.slice/crio-e11f7e0e5e7e3aba963c2174ecd6876839e1c837554f32c1a471d308f033fd9a WatchSource:0}: Error finding container e11f7e0e5e7e3aba963c2174ecd6876839e1c837554f32c1a471d308f033fd9a: Status 404 returned error can't find the container with id e11f7e0e5e7e3aba963c2174ecd6876839e1c837554f32c1a471d308f033fd9a Jul 15 21:30:36.873772 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.873735 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:36.967138 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.967069 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:36.972877 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.972708 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 21:30:36.978620 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.978570 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:36.981331 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.981309 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 21:30:36.982295 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.982268 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 21:30:36.982508 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.982489 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 21:30:36.982704 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.982687 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 21:30:36.982897 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.982881 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 21:30:36.983297 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.983278 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-47mtv\"" Jul 15 21:30:36.983513 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.983494 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 21:30:36.984700 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.984678 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 21:30:36.984924 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.984906 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 21:30:36.987005 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.986979 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 21:30:36.998873 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:36.998851 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 21:30:37.000001 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:36.999965 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8ad9669_bb51_45d5_82dd_33a17a08a991.slice/crio-517e0afc8b2f30e501dcf252edc14412ec04107becbef023fa6efddae2e08fc1 WatchSource:0}: Error finding container 517e0afc8b2f30e501dcf252edc14412ec04107becbef023fa6efddae2e08fc1: Status 404 returned error can't find the container with id 517e0afc8b2f30e501dcf252edc14412ec04107becbef023fa6efddae2e08fc1 Jul 15 21:30:37.000696 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.000673 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng"] Jul 15 21:30:37.012591 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.012591 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012551 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.012775 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012624 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.012775 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.012775 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012703 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.012775 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012732 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.012775 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012757 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-web-config\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.013007 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012803 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.013007 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012828 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.013007 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.013007 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012883 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-out\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.013007 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012913 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.013007 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.012941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5rns\" (UniqueName: \"kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-kube-api-access-k5rns\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.030919 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.030875 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-7647x"] Jul 15 21:30:37.036836 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:37.036803 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c473fde_ec1e_436a_acf6_c3cf25bd25b7.slice/crio-2bb8eb07d58e1e888742952493338d569ef79818482528f921a66531fb56610e WatchSource:0}: Error finding container 2bb8eb07d58e1e888742952493338d569ef79818482528f921a66531fb56610e: Status 404 returned error can't find the container with id 2bb8eb07d58e1e888742952493338d569ef79818482528f921a66531fb56610e Jul 15 21:30:37.113503 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113503 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113451 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113503 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113478 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113833 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113833 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113534 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-web-config\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113833 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113577 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113833 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113601 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113833 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113833 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113650 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-out\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113833 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113833 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5rns\" (UniqueName: \"kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-kube-api-access-k5rns\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113833 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113770 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.113833 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113792 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.114560 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.113931 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.114609 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:37.113579 2578 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 15 21:30:37.114723 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:37.113955 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-trusted-ca-bundle podName:b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7 nodeName:}" failed. No retries permitted until 2026-07-15 21:30:37.613930956 +0000 UTC m=+79.117366242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7") : configmap references non-existent config key: ca-bundle.crt Jul 15 21:30:37.114723 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:30:37.114677 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-main-tls podName:b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7 nodeName:}" failed. No retries permitted until 2026-07-15 21:30:37.61465802 +0000 UTC m=+79.118093297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7") : secret "alertmanager-main-tls" not found Jul 15 21:30:37.115317 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.115271 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.116985 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.116942 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-out\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.117073 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.117027 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.118653 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.118592 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-web-config\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.118653 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.118619 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.118812 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.118718 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.118938 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.118909 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.120134 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.120067 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.120426 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.120405 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.127647 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.127617 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5rns\" (UniqueName: \"kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-kube-api-access-k5rns\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.317841 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.317800 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:37.415859 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.415761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sggl4" event={"ID":"71b48e14-1112-4ec7-a791-59a3bcb638fd","Type":"ContainerStarted","Data":"e11f7e0e5e7e3aba963c2174ecd6876839e1c837554f32c1a471d308f033fd9a"} Jul 15 21:30:37.417153 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.417117 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" event={"ID":"1c473fde-ec1e-436a-acf6-c3cf25bd25b7","Type":"ContainerStarted","Data":"2bb8eb07d58e1e888742952493338d569ef79818482528f921a66531fb56610e"} Jul 15 21:30:37.419191 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.419155 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" event={"ID":"e8ad9669-bb51-45d5-82dd-33a17a08a991","Type":"ContainerStarted","Data":"3026a0a9ac3ce9a64cbf664e57eb24af751c37d6d3a5058988b7016f32baade2"} Jul 15 21:30:37.419312 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.419206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" event={"ID":"e8ad9669-bb51-45d5-82dd-33a17a08a991","Type":"ContainerStarted","Data":"e822832c3082cdf4fb1aae1cddc28c4768cb11ab0fdbc7aef604c6094aae33ed"} Jul 15 21:30:37.419312 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.419228 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" event={"ID":"e8ad9669-bb51-45d5-82dd-33a17a08a991","Type":"ContainerStarted","Data":"517e0afc8b2f30e501dcf252edc14412ec04107becbef023fa6efddae2e08fc1"} Jul 15 21:30:37.617220 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.617127 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.617389 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.617256 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.618297 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.618240 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.621783 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.621750 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.900920 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.900823 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:30:37.957520 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.957486 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs"] Jul 15 21:30:37.961359 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.961332 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:37.963881 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.963768 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 15 21:30:37.963881 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.963798 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 15 21:30:37.963881 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.963771 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-d427l\"" Jul 15 21:30:37.963881 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.963849 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 15 21:30:37.964174 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.964047 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 15 21:30:37.964174 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.964073 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 15 21:30:37.964329 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.964314 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cfg3srl6l80s3\"" Jul 15 21:30:37.976725 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:37.973139 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs"] Jul 15 21:30:38.020643 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.020603 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.020643 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.020647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eb8d285-7470-48fe-bbf0-2d52824af9bf-metrics-client-ca\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.020889 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.020679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-tls\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.020889 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.020748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.020889 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.020831 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.020889 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.020876 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.021123 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.020949 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-grpc-tls\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.021123 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.020974 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctrmd\" (UniqueName: \"kubernetes.io/projected/3eb8d285-7470-48fe-bbf0-2d52824af9bf-kube-api-access-ctrmd\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.122337 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.122296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.122337 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.122340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eb8d285-7470-48fe-bbf0-2d52824af9bf-metrics-client-ca\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.122564 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.122368 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-tls\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.122564 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.122404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.122564 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.122459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.122564 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.122507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.122564 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.122558 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-grpc-tls\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.122782 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.122594 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctrmd\" (UniqueName: \"kubernetes.io/projected/3eb8d285-7470-48fe-bbf0-2d52824af9bf-kube-api-access-ctrmd\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.123949 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.123891 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eb8d285-7470-48fe-bbf0-2d52824af9bf-metrics-client-ca\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.125785 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.125737 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.125901 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.125883 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-grpc-tls\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.126257 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.126232 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.126364 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.126309 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.127814 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.127766 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-tls\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.128562 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.128532 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3eb8d285-7470-48fe-bbf0-2d52824af9bf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.130921 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.130887 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctrmd\" (UniqueName: \"kubernetes.io/projected/3eb8d285-7470-48fe-bbf0-2d52824af9bf-kube-api-access-ctrmd\") pod \"thanos-querier-7c54bc95ff-ll9hs\" (UID: \"3eb8d285-7470-48fe-bbf0-2d52824af9bf\") " pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.275799 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.275713 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:38.466296 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.466211 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:38.524711 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:38.524674 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:39.056863 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.056831 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs"] Jul 15 21:30:39.079228 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.079202 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 21:30:39.086719 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:39.086264 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1bc9c18_dcf0_456c_a003_8d8f21a4e7b7.slice/crio-c4c821f56176cdaa2664e1e58a0d482e14cc4574d3f4211d05f9d6bb62b55af4 WatchSource:0}: Error finding container c4c821f56176cdaa2664e1e58a0d482e14cc4574d3f4211d05f9d6bb62b55af4: Status 404 returned error can't find the container with id c4c821f56176cdaa2664e1e58a0d482e14cc4574d3f4211d05f9d6bb62b55af4 Jul 15 21:30:39.225912 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.225850 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:39.322923 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.322833 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:39.382816 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.382780 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:39.430125 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.430066 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" event={"ID":"3eb8d285-7470-48fe-bbf0-2d52824af9bf","Type":"ContainerStarted","Data":"6bd9b2d09ed61f544026e3989689045983bf39e6ffb5c6c6d4e7e758e864283f"} Jul 15 21:30:39.431643 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.431612 2578 generic.go:356] "Generic (PLEG): container finished" podID="71b48e14-1112-4ec7-a791-59a3bcb638fd" containerID="0c29d5652166f4c363ebe7d16d7056b4c788d337aa0c662b8fa8b05b47aecfa8" exitCode=0 Jul 15 21:30:39.431766 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.431695 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sggl4" event={"ID":"71b48e14-1112-4ec7-a791-59a3bcb638fd","Type":"ContainerDied","Data":"0c29d5652166f4c363ebe7d16d7056b4c788d337aa0c662b8fa8b05b47aecfa8"} Jul 15 21:30:39.438144 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.438119 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" event={"ID":"1c473fde-ec1e-436a-acf6-c3cf25bd25b7","Type":"ContainerStarted","Data":"ee072d776bb586b197a52fa85baff28c91d1d340986c59cbd30b8c7c9e11519a"} Jul 15 21:30:39.438273 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.438179 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" event={"ID":"1c473fde-ec1e-436a-acf6-c3cf25bd25b7","Type":"ContainerStarted","Data":"6d2e1bbb3c02f5e86eb2a52ac5ca92419ee868344460731b1a312b24a2fcd91a"} Jul 15 21:30:39.438273 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.438209 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" event={"ID":"1c473fde-ec1e-436a-acf6-c3cf25bd25b7","Type":"ContainerStarted","Data":"b142dc2167bf3001dc42b05d7e297bcdab07a589a43a7f8531fba79a88dcca7c"} Jul 15 21:30:39.441053 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.441028 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" event={"ID":"e8ad9669-bb51-45d5-82dd-33a17a08a991","Type":"ContainerStarted","Data":"6bd3bc78cf3e384c39a3eae09f95133691adf223f4e027d842a6532ee9b9a344"} Jul 15 21:30:39.442394 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.442352 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerStarted","Data":"c4c821f56176cdaa2664e1e58a0d482e14cc4574d3f4211d05f9d6bb62b55af4"} Jul 15 21:30:39.479186 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.479085 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-7647x" podStartSLOduration=2.687600522 podStartE2EDuration="4.479067108s" totalImagesPullingTime="1.791466586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:39.478036891 +0000 UTC m=+80.981472183" watchObservedRunningTime="2026-07-15 21:30:39.479067108 +0000 UTC m=+80.982502402" Jul 15 21:30:39.495428 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.495375 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-ck4ng" podStartSLOduration=2.8037803439999998 podStartE2EDuration="4.495354904s" totalImagesPullingTime="1.69157456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:39.494675272 +0000 UTC m=+80.998110568" watchObservedRunningTime="2026-07-15 21:30:39.495354904 +0000 UTC m=+80.998790201" Jul 15 21:30:39.572607 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.572539 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:30:39.727129 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.727069 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:39.828789 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.828697 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:39.929656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:39.929615 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:40.324234 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.323200 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-d7bb8877b-hfmt8"] Jul 15 21:30:40.327397 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.326879 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.329768 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.329603 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 15 21:30:40.329868 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.329767 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 15 21:30:40.331206 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.331120 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-x6v57\"" Jul 15 21:30:40.331404 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.331379 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 15 21:30:40.331642 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.331620 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 15 21:30:40.331835 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.331817 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8vb632n6gqa4v\"" Jul 15 21:30:40.335561 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.335498 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d7bb8877b-hfmt8"] Jul 15 21:30:40.352034 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.351544 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4sld\" (UniqueName: \"kubernetes.io/projected/3bd4b704-c13e-41fc-b974-0abb041423da-kube-api-access-s4sld\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.352034 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.351590 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3bd4b704-c13e-41fc-b974-0abb041423da-audit-log\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.352034 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.351617 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3bd4b704-c13e-41fc-b974-0abb041423da-client-ca-bundle\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.352034 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.351698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bd4b704-c13e-41fc-b974-0abb041423da-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.352034 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.351735 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3bd4b704-c13e-41fc-b974-0abb041423da-secret-metrics-server-client-certs\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.352034 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.351775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3bd4b704-c13e-41fc-b974-0abb041423da-metrics-server-audit-profiles\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.352034 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.351801 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3bd4b704-c13e-41fc-b974-0abb041423da-secret-metrics-server-tls\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.452286 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.452250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bd4b704-c13e-41fc-b974-0abb041423da-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.452286 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.452294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3bd4b704-c13e-41fc-b974-0abb041423da-secret-metrics-server-client-certs\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.452532 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.452330 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3bd4b704-c13e-41fc-b974-0abb041423da-metrics-server-audit-profiles\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.452532 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.452355 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3bd4b704-c13e-41fc-b974-0abb041423da-secret-metrics-server-tls\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.452532 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.452416 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4sld\" (UniqueName: \"kubernetes.io/projected/3bd4b704-c13e-41fc-b974-0abb041423da-kube-api-access-s4sld\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.452532 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.452450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3bd4b704-c13e-41fc-b974-0abb041423da-audit-log\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.452532 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.452470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3bd4b704-c13e-41fc-b974-0abb041423da-client-ca-bundle\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.454106 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.453055 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sggl4" event={"ID":"71b48e14-1112-4ec7-a791-59a3bcb638fd","Type":"ContainerStarted","Data":"d894abf26544e1edd6f4bf37115678cc51019a050eefe26b46a23a9d7898c348"} Jul 15 21:30:40.454106 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.453137 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sggl4" event={"ID":"71b48e14-1112-4ec7-a791-59a3bcb638fd","Type":"ContainerStarted","Data":"76f204a1a2dee9e3c6fe312532c507436ede62bfcfd170d367cdb0448eab0917"} Jul 15 21:30:40.454106 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.453996 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bd4b704-c13e-41fc-b974-0abb041423da-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.454106 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.454026 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3bd4b704-c13e-41fc-b974-0abb041423da-metrics-server-audit-profiles\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.454644 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.454620 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3bd4b704-c13e-41fc-b974-0abb041423da-audit-log\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.455964 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.455938 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3bd4b704-c13e-41fc-b974-0abb041423da-client-ca-bundle\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.458287 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.458265 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3bd4b704-c13e-41fc-b974-0abb041423da-secret-metrics-server-client-certs\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.458391 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.458344 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3bd4b704-c13e-41fc-b974-0abb041423da-secret-metrics-server-tls\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.461314 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.461270 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4sld\" (UniqueName: \"kubernetes.io/projected/3bd4b704-c13e-41fc-b974-0abb041423da-kube-api-access-s4sld\") pod \"metrics-server-d7bb8877b-hfmt8\" (UID: \"3bd4b704-c13e-41fc-b974-0abb041423da\") " pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.475272 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.474631 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sggl4" podStartSLOduration=3.345033423 podStartE2EDuration="5.474608356s" totalImagesPullingTime="1.96873422s" totalInitContainerRuntime="160.840713ms" isStatefulPod=true podCreationTimestamp="2026-07-15 21:30:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:40.474568092 +0000 UTC m=+81.978003390" watchObservedRunningTime="2026-07-15 21:30:40.474608356 +0000 UTC m=+81.978043649" Jul 15 21:30:40.505783 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.505743 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:40.559198 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.559164 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:40.644887 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.644538 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:30:40.662584 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.658725 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:40.670242 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.669119 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4"] Jul 15 21:30:40.673955 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.673929 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" Jul 15 21:30:40.677131 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.677080 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 15 21:30:40.677280 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.677255 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-zjjbm\"" Jul 15 21:30:40.683132 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.683031 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4"] Jul 15 21:30:40.719786 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.719722 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:40.756065 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.755994 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97e3a6fb-d8e6-4979-91d7-a45c65aac66e-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-5lkd4\" (UID: \"97e3a6fb-d8e6-4979-91d7-a45c65aac66e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" Jul 15 21:30:40.756065 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.756050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wppn7\" (UniqueName: \"kubernetes.io/projected/97e3a6fb-d8e6-4979-91d7-a45c65aac66e-kube-api-access-wppn7\") pod \"monitoring-plugin-5f68df7f58-5lkd4\" (UID: \"97e3a6fb-d8e6-4979-91d7-a45c65aac66e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" Jul 15 21:30:40.823685 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.823639 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d7bb8877b-hfmt8"] Jul 15 21:30:40.830562 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.830431 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:40.857580 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.857546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97e3a6fb-d8e6-4979-91d7-a45c65aac66e-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-5lkd4\" (UID: \"97e3a6fb-d8e6-4979-91d7-a45c65aac66e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" Jul 15 21:30:40.857739 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.857593 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wppn7\" (UniqueName: \"kubernetes.io/projected/97e3a6fb-d8e6-4979-91d7-a45c65aac66e-kube-api-access-wppn7\") pod \"monitoring-plugin-5f68df7f58-5lkd4\" (UID: \"97e3a6fb-d8e6-4979-91d7-a45c65aac66e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" Jul 15 21:30:40.860621 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.860568 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97e3a6fb-d8e6-4979-91d7-a45c65aac66e-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-5lkd4\" (UID: \"97e3a6fb-d8e6-4979-91d7-a45c65aac66e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" Jul 15 21:30:40.866374 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.866322 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wppn7\" (UniqueName: \"kubernetes.io/projected/97e3a6fb-d8e6-4979-91d7-a45c65aac66e-kube-api-access-wppn7\") pod \"monitoring-plugin-5f68df7f58-5lkd4\" (UID: \"97e3a6fb-d8e6-4979-91d7-a45c65aac66e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" Jul 15 21:30:40.992614 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:40.992518 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" Jul 15 21:30:41.011172 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:41.011128 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd4b704_c13e_41fc_b974_0abb041423da.slice/crio-b735eb01cb7846f8f1135db5017dd1f7551954fe5d83d363751bc257c7751a00 WatchSource:0}: Error finding container b735eb01cb7846f8f1135db5017dd1f7551954fe5d83d363751bc257c7751a00: Status 404 returned error can't find the container with id b735eb01cb7846f8f1135db5017dd1f7551954fe5d83d363751bc257c7751a00 Jul 15 21:30:41.337549 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:41.337515 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59d7c4c88f-z5dc7" Jul 15 21:30:41.453975 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:41.453936 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:41.459076 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:41.459036 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" event={"ID":"3bd4b704-c13e-41fc-b974-0abb041423da","Type":"ContainerStarted","Data":"b735eb01cb7846f8f1135db5017dd1f7551954fe5d83d363751bc257c7751a00"} Jul 15 21:30:41.554304 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:41.554216 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:41.700368 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:41.699962 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:41.798430 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:41.798396 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:41.840370 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:41.840347 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4"] Jul 15 21:30:41.870835 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:41.870801 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e3a6fb_d8e6_4979_91d7_a45c65aac66e.slice/crio-0bd60f0f211527f4ce16f0b302a9eb30db849575a882a02be831169fa1c054be WatchSource:0}: Error finding container 0bd60f0f211527f4ce16f0b302a9eb30db849575a882a02be831169fa1c054be: Status 404 returned error can't find the container with id 0bd60f0f211527f4ce16f0b302a9eb30db849575a882a02be831169fa1c054be Jul 15 21:30:42.074789 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.074407 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 21:30:42.079284 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.079254 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.081917 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.081888 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 15 21:30:42.083173 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.082191 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 15 21:30:42.083173 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.082302 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 15 21:30:42.083173 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.082383 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 15 21:30:42.083173 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.082468 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 15 21:30:42.083173 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.082687 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-l8sfg\"" Jul 15 21:30:42.083173 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.082693 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b3u7579tff5k\"" Jul 15 21:30:42.083173 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.082845 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 15 21:30:42.083173 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.082904 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 15 21:30:42.083173 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.082952 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 15 21:30:42.083173 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.082902 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 15 21:30:42.083173 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.083081 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 15 21:30:42.084145 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.083233 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 15 21:30:42.084145 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.083487 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 15 21:30:42.085229 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.084924 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 15 21:30:42.090192 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.090168 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 15 21:30:42.090579 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.090555 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" limit=1024 Jul 15 21:30:42.095629 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.095520 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 21:30:42.174356 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174316 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174532 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174372 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174532 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174532 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174532 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174447 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174532 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174475 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174532 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174802 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174607 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174802 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174802 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174743 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174802 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174802 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174800 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174999 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174825 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-web-config\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174999 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174861 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174999 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174876 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhkkk\" (UniqueName: \"kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-kube-api-access-bhkkk\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174999 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-config\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174999 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.174999 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.174963 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.175280 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.175017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-config-out\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.175280 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.175067 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.276278 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.276189 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.276453 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.276277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.276453 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.276303 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.276453 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.276328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.276453 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.276357 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-web-config\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.276453 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.276398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.276453 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.276423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhkkk\" (UniqueName: \"kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-kube-api-access-bhkkk\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.276453 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.276455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-config\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.277393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.277393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277076 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.277393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-config-out\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.277393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.277393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277208 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.277393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277236 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.277393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.277393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.277393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.277393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.277393 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277372 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.278394 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277406 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.278394 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.277546 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.278394 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.278164 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.279126 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.278625 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.279623 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.279594 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.280199 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.280177 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.280402 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.280382 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-config\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.280723 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.280702 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.280813 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.280792 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.282461 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.281766 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.282461 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.281858 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-web-config\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.282461 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.282126 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.282461 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.282321 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.282461 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.282395 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.283295 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.283269 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.283545 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.283522 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.283715 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.283688 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.283951 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.283934 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.284882 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.284859 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.285622 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.285598 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-config-out\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.285821 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.285782 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhkkk\" (UniqueName: \"kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-kube-api-access-bhkkk\") pod \"prometheus-k8s-0\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.393905 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.393871 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:30:42.472627 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.472397 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" event={"ID":"3eb8d285-7470-48fe-bbf0-2d52824af9bf","Type":"ContainerStarted","Data":"847b158bfd0c23f47ab5e433be2af3464ec8f2f1234951d80da5d35656791a6e"} Jul 15 21:30:42.472627 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.472516 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" event={"ID":"3eb8d285-7470-48fe-bbf0-2d52824af9bf","Type":"ContainerStarted","Data":"db6bcc02ce8d97ef9afc399594efcba1860ff8501178473e344481f5f8657560"} Jul 15 21:30:42.472627 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.472561 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" event={"ID":"3eb8d285-7470-48fe-bbf0-2d52824af9bf","Type":"ContainerStarted","Data":"fa9259b9e2573996b386a0b166946cf78b6405ca28f8e14cd31f165bd7f60e0e"} Jul 15 21:30:42.476729 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.476648 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" event={"ID":"97e3a6fb-d8e6-4979-91d7-a45c65aac66e","Type":"ContainerStarted","Data":"0bd60f0f211527f4ce16f0b302a9eb30db849575a882a02be831169fa1c054be"} Jul 15 21:30:42.481004 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.480951 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerID="bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d" exitCode=0 Jul 15 21:30:42.481071 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.481007 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerDied","Data":"bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d"} Jul 15 21:30:42.582917 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:42.581614 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 21:30:42.586318 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:30:42.586208 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda699d543_58a5_4dc4_93d3_b52343415a86.slice/crio-4bebc8735f97af265520854a1cfc6dd551484ccc625a85773f23120a96b3cb55 WatchSource:0}: Error finding container 4bebc8735f97af265520854a1cfc6dd551484ccc625a85773f23120a96b3cb55: Status 404 returned error can't find the container with id 4bebc8735f97af265520854a1cfc6dd551484ccc625a85773f23120a96b3cb55 Jul 15 21:30:43.489240 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:43.489204 2578 generic.go:356] "Generic (PLEG): container finished" podID="a699d543-58a5-4dc4-93d3-b52343415a86" containerID="4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500" exitCode=0 Jul 15 21:30:43.489603 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:43.489285 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerDied","Data":"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500"} Jul 15 21:30:43.489603 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:43.489354 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerStarted","Data":"4bebc8735f97af265520854a1cfc6dd551484ccc625a85773f23120a96b3cb55"} Jul 15 21:30:44.595329 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:44.595283 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" podUID="99581639-234d-413f-be81-2ba54826a4cf" containerName="registry" containerID="cri-o://8a4698749be7ede2efad5bc9f1e2685efc4d2e44a707ce05a8f5c604ee6cd429" gracePeriod=30 Jul 15 21:30:49.568171 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:49.568133 2578 patch_prober.go:28] interesting pod/image-registry-6dcd789698-x8rkr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.133.0.6:5000/healthz\": dial tcp 10.133.0.6:5000: connect: connection refused" start-of-body= Jul 15 21:30:49.568669 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:49.568193 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" podUID="99581639-234d-413f-be81-2ba54826a4cf" containerName="registry" probeResult="failure" output="Get \"https://10.133.0.6:5000/healthz\": dial tcp 10.133.0.6:5000: connect: connection refused" Jul 15 21:30:51.533982 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.533903 2578 generic.go:356] "Generic (PLEG): container finished" podID="99581639-234d-413f-be81-2ba54826a4cf" containerID="8a4698749be7ede2efad5bc9f1e2685efc4d2e44a707ce05a8f5c604ee6cd429" exitCode=0 Jul 15 21:30:51.533982 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.533951 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" event={"ID":"99581639-234d-413f-be81-2ba54826a4cf","Type":"ContainerDied","Data":"8a4698749be7ede2efad5bc9f1e2685efc4d2e44a707ce05a8f5c604ee6cd429"} Jul 15 21:30:51.576791 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.576758 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:30:51.669271 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.669233 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-trusted-ca\") pod \"99581639-234d-413f-be81-2ba54826a4cf\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " Jul 15 21:30:51.669468 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.669305 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/99581639-234d-413f-be81-2ba54826a4cf-ca-trust-extracted\") pod \"99581639-234d-413f-be81-2ba54826a4cf\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " Jul 15 21:30:51.669468 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.669335 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls\") pod \"99581639-234d-413f-be81-2ba54826a4cf\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " Jul 15 21:30:51.669468 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.669366 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnnz9\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-kube-api-access-rnnz9\") pod \"99581639-234d-413f-be81-2ba54826a4cf\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " Jul 15 21:30:51.669468 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.669405 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-installation-pull-secrets\") pod \"99581639-234d-413f-be81-2ba54826a4cf\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " Jul 15 21:30:51.669681 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.669488 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-image-registry-private-configuration\") pod \"99581639-234d-413f-be81-2ba54826a4cf\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " Jul 15 21:30:51.669681 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.669524 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-registry-certificates\") pod \"99581639-234d-413f-be81-2ba54826a4cf\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " Jul 15 21:30:51.669681 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.669563 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-bound-sa-token\") pod \"99581639-234d-413f-be81-2ba54826a4cf\" (UID: \"99581639-234d-413f-be81-2ba54826a4cf\") " Jul 15 21:30:51.669811 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.669720 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "99581639-234d-413f-be81-2ba54826a4cf" (UID: "99581639-234d-413f-be81-2ba54826a4cf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:30:51.670004 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.669972 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-trusted-ca\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:30:51.670149 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.670082 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "99581639-234d-413f-be81-2ba54826a4cf" (UID: "99581639-234d-413f-be81-2ba54826a4cf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:30:51.672478 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.672444 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-kube-api-access-rnnz9" (OuterVolumeSpecName: "kube-api-access-rnnz9") pod "99581639-234d-413f-be81-2ba54826a4cf" (UID: "99581639-234d-413f-be81-2ba54826a4cf"). InnerVolumeSpecName "kube-api-access-rnnz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:30:51.672699 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.672660 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "99581639-234d-413f-be81-2ba54826a4cf" (UID: "99581639-234d-413f-be81-2ba54826a4cf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:30:51.672906 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.672810 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "99581639-234d-413f-be81-2ba54826a4cf" (UID: "99581639-234d-413f-be81-2ba54826a4cf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:30:51.673143 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.673119 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "99581639-234d-413f-be81-2ba54826a4cf" (UID: "99581639-234d-413f-be81-2ba54826a4cf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:30:51.673721 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.673689 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "99581639-234d-413f-be81-2ba54826a4cf" (UID: "99581639-234d-413f-be81-2ba54826a4cf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:30:51.680151 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.680125 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99581639-234d-413f-be81-2ba54826a4cf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "99581639-234d-413f-be81-2ba54826a4cf" (UID: "99581639-234d-413f-be81-2ba54826a4cf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:30:51.771353 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.771312 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-image-registry-private-configuration\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:30:51.771353 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.771348 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/99581639-234d-413f-be81-2ba54826a4cf-registry-certificates\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:30:51.771353 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.771359 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-bound-sa-token\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:30:51.771649 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.771367 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/99581639-234d-413f-be81-2ba54826a4cf-ca-trust-extracted\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:30:51.771649 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.771378 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-registry-tls\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:30:51.771649 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.771386 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnnz9\" (UniqueName: \"kubernetes.io/projected/99581639-234d-413f-be81-2ba54826a4cf-kube-api-access-rnnz9\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:30:51.771649 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:51.771394 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/99581639-234d-413f-be81-2ba54826a4cf-installation-pull-secrets\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:30:52.542831 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.542798 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerStarted","Data":"d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94"} Jul 15 21:30:52.546358 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.546243 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" event={"ID":"3eb8d285-7470-48fe-bbf0-2d52824af9bf","Type":"ContainerStarted","Data":"7e65930da90835c618a3f73e7c2ef3ea07a4f5c5a799e3c5872ed25a4758eae6"} Jul 15 21:30:52.548432 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.548403 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" event={"ID":"3bd4b704-c13e-41fc-b974-0abb041423da","Type":"ContainerStarted","Data":"dca0359d3fe752b261560c1bdfe9b603b996073e9098203ac87fb375127d44e1"} Jul 15 21:30:52.549912 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.549889 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" event={"ID":"99581639-234d-413f-be81-2ba54826a4cf","Type":"ContainerDied","Data":"1a1a8af3e58f1b90df2fa27672530a4d6d17bf773aff700266d8480e72d5e02f"} Jul 15 21:30:52.550018 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.549930 2578 scope.go:117] "RemoveContainer" containerID="8a4698749be7ede2efad5bc9f1e2685efc4d2e44a707ce05a8f5c604ee6cd429" Jul 15 21:30:52.550075 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.550056 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcd789698-x8rkr" Jul 15 21:30:52.553386 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.553107 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-z6vvf" event={"ID":"ef049efd-fdaa-48eb-9766-9c4647e076c0","Type":"ContainerStarted","Data":"117b35c5a5fe95225265902f069ab40b1f6267a8f788da84b66c05a53fd7e1ce"} Jul 15 21:30:52.555965 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.555938 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" event={"ID":"97e3a6fb-d8e6-4979-91d7-a45c65aac66e","Type":"ContainerStarted","Data":"025db4f4e05519746fa252338fd12ac21de62774511603a2aff03ea95e818508"} Jul 15 21:30:52.561321 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.561016 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-z6vvf" Jul 15 21:30:52.562112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.562067 2578 patch_prober.go:28] interesting pod/downloads-68c9cbb497-z6vvf container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.17:8080/\": dial tcp 10.133.0.17:8080: connect: connection refused" start-of-body= Jul 15 21:30:52.562199 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.562129 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-z6vvf" podUID="ef049efd-fdaa-48eb-9766-9c4647e076c0" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.17:8080/\": dial tcp 10.133.0.17:8080: connect: connection refused" Jul 15 21:30:52.580963 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.580911 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-z6vvf" podStartSLOduration=2.913946433 podStartE2EDuration="21.58088998s" totalImagesPullingTime="18.666943547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:52.576809996 +0000 UTC m=+94.080245295" watchObservedRunningTime="2026-07-15 21:30:52.58088998 +0000 UTC m=+94.084325274" Jul 15 21:30:52.595029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.594977 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" podStartSLOduration=2.070411041 podStartE2EDuration="12.594954778s" totalImagesPullingTime="10.524543737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:52.593404388 +0000 UTC m=+94.096839683" watchObservedRunningTime="2026-07-15 21:30:52.594954778 +0000 UTC m=+94.098390077" Jul 15 21:30:52.610473 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.610431 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dcd789698-x8rkr"] Jul 15 21:30:52.616649 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.616619 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6dcd789698-x8rkr"] Jul 15 21:30:52.630436 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:52.630379 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" podStartSLOduration=2.556260681 podStartE2EDuration="12.630355212s" totalImagesPullingTime="10.074094531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:52.628183203 +0000 UTC m=+94.131618500" watchObservedRunningTime="2026-07-15 21:30:52.630355212 +0000 UTC m=+94.133790509" Jul 15 21:30:53.055515 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.055288 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99581639-234d-413f-be81-2ba54826a4cf" path="/var/lib/kubelet/pods/99581639-234d-413f-be81-2ba54826a4cf/volumes" Jul 15 21:30:53.569754 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.569707 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerStarted","Data":"0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e"} Jul 15 21:30:53.570357 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.569837 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerStarted","Data":"58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0"} Jul 15 21:30:53.570357 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.569880 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerStarted","Data":"1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1"} Jul 15 21:30:53.570357 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.569923 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerStarted","Data":"d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310"} Jul 15 21:30:53.570357 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.569960 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerStarted","Data":"6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556"} Jul 15 21:30:53.574488 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.573731 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" event={"ID":"3eb8d285-7470-48fe-bbf0-2d52824af9bf","Type":"ContainerStarted","Data":"2c871a2167636d5fdfb6270fa8c1c8725701aa765c2ce42c1f4c987743045c36"} Jul 15 21:30:53.574488 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.573825 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" event={"ID":"3eb8d285-7470-48fe-bbf0-2d52824af9bf","Type":"ContainerStarted","Data":"96cc7b714820ef08b404ca2063f658e3008d392e28f3605bcb9bdeed92ee4357"} Jul 15 21:30:53.574488 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.573872 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:53.576852 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.576830 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" Jul 15 21:30:53.580326 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.580291 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" Jul 15 21:30:53.581038 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.581016 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5lkd4" Jul 15 21:30:53.588047 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.588013 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-z6vvf" Jul 15 21:30:53.601298 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.601242 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.139667382 podStartE2EDuration="17.601223958s" totalImagesPullingTime="12.436557291s" totalInitContainerRuntime="24.999285ms" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:53.59894812 +0000 UTC m=+95.102383411" watchObservedRunningTime="2026-07-15 21:30:53.601223958 +0000 UTC m=+95.104659252" Jul 15 21:30:53.623418 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:53.621496 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7c54bc95ff-ll9hs" podStartSLOduration=3.655436668 podStartE2EDuration="16.621471152s" totalImagesPullingTime="12.966034484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:53.619472821 +0000 UTC m=+95.122908152" watchObservedRunningTime="2026-07-15 21:30:53.621471152 +0000 UTC m=+95.124906447" Jul 15 21:30:56.601722 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:56.601617 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerStarted","Data":"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2"} Jul 15 21:30:56.601722 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:56.601717 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerStarted","Data":"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a"} Jul 15 21:30:56.602352 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:56.601771 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerStarted","Data":"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988"} Jul 15 21:30:56.602352 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:56.601816 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerStarted","Data":"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458"} Jul 15 21:30:56.602352 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:56.601857 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerStarted","Data":"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb"} Jul 15 21:30:56.602352 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:56.601901 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerStarted","Data":"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f"} Jul 15 21:30:56.635022 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:56.632774 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.588339769 podStartE2EDuration="14.632750202s" totalImagesPullingTime="12.016945727s" totalInitContainerRuntime="27.464706ms" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:56.628981671 +0000 UTC m=+98.132416966" watchObservedRunningTime="2026-07-15 21:30:56.632750202 +0000 UTC m=+98.136185497" Jul 15 21:30:57.394505 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:30:57.394457 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:31:00.645884 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:00.645838 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:31:00.646389 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:00.645961 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:31:03.398343 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:03.398316 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-t9g8r" Jul 15 21:31:20.650694 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:20.650657 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:31:20.654807 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:20.654753 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-d7bb8877b-hfmt8" Jul 15 21:31:42.394347 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:42.394312 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:31:42.450168 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:42.450139 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:31:42.831444 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:42.831416 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:31:56.256733 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.256698 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 21:31:56.257316 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.257009 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="alertmanager" containerID="cri-o://d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94" gracePeriod=120 Jul 15 21:31:56.257316 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.257045 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy-metric" containerID="cri-o://58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0" gracePeriod=120 Jul 15 21:31:56.257316 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.257133 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy-web" containerID="cri-o://d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310" gracePeriod=120 Jul 15 21:31:56.257316 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.257188 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="config-reloader" containerID="cri-o://6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556" gracePeriod=120 Jul 15 21:31:56.257561 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.257370 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="prom-label-proxy" containerID="cri-o://0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e" gracePeriod=120 Jul 15 21:31:56.257561 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.257417 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy" containerID="cri-o://1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1" gracePeriod=120 Jul 15 21:31:56.878029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.877993 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerID="0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e" exitCode=0 Jul 15 21:31:56.878029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.878021 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerID="1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1" exitCode=0 Jul 15 21:31:56.878029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.878027 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerID="6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556" exitCode=0 Jul 15 21:31:56.878029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.878033 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerID="d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94" exitCode=0 Jul 15 21:31:56.878339 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.878066 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerDied","Data":"0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e"} Jul 15 21:31:56.878339 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.878163 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerDied","Data":"1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1"} Jul 15 21:31:56.878339 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.878192 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerDied","Data":"6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556"} Jul 15 21:31:56.878339 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:56.878217 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerDied","Data":"d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94"} Jul 15 21:31:57.512021 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.511993 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:31:57.675112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.674974 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5rns\" (UniqueName: \"kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-kube-api-access-k5rns\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675033 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-tls-assets\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675058 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675112 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675120 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675456 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675267 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-web-config\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675456 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675328 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-main-tls\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675456 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675356 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-volume\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675456 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675392 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-cluster-tls-config\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675456 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675422 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-trusted-ca-bundle\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675713 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675488 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-web\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675713 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675519 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-main-db\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675713 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675541 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-metrics-client-ca\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.675713 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.675599 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-out\") pod \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\" (UID: \"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7\") " Jul 15 21:31:57.677386 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.677350 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:57.677858 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.677818 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:31:57.678006 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.677853 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:31:57.678006 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.677899 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:31:57.678178 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.678143 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:31:57.678331 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.678301 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-kube-api-access-k5rns" (OuterVolumeSpecName: "kube-api-access-k5rns") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "kube-api-access-k5rns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:31:57.678331 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.678302 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:31:57.678553 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.678518 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-volume" (OuterVolumeSpecName: "config-volume") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:31:57.678952 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.678926 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:31:57.679175 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.679068 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-out" (OuterVolumeSpecName: "config-out") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:57.679936 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.679906 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:31:57.683585 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.683558 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:31:57.691214 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.691182 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-web-config" (OuterVolumeSpecName: "web-config") pod "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" (UID: "b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:31:57.776812 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776763 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-main-tls\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.776812 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776807 2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-volume\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.776812 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776818 2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-cluster-tls-config\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.776812 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776829 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.777068 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776839 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.777068 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776848 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-alertmanager-main-db\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.777068 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776858 2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-metrics-client-ca\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.777068 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776866 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-config-out\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.777068 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776874 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5rns\" (UniqueName: \"kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-kube-api-access-k5rns\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.777068 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776882 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-tls-assets\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.777068 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776890 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.777068 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776899 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.777068 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.776909 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7-web-config\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:31:57.885749 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.885716 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerID="58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0" exitCode=0 Jul 15 21:31:57.885749 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.885746 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerID="d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310" exitCode=0 Jul 15 21:31:57.885934 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.885792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerDied","Data":"58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0"} Jul 15 21:31:57.885934 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.885861 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerDied","Data":"d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310"} Jul 15 21:31:57.885934 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.885806 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:31:57.885934 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.885889 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7","Type":"ContainerDied","Data":"c4c821f56176cdaa2664e1e58a0d482e14cc4574d3f4211d05f9d6bb62b55af4"} Jul 15 21:31:57.885934 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.885921 2578 scope.go:117] "RemoveContainer" containerID="0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e" Jul 15 21:31:57.896972 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.895405 2578 scope.go:117] "RemoveContainer" containerID="58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0" Jul 15 21:31:57.905040 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.905020 2578 scope.go:117] "RemoveContainer" containerID="1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1" Jul 15 21:31:57.912445 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.912410 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 21:31:57.913849 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.913824 2578 scope.go:117] "RemoveContainer" containerID="d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310" Jul 15 21:31:57.916127 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.916106 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 21:31:57.922108 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.922071 2578 scope.go:117] "RemoveContainer" containerID="6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556" Jul 15 21:31:57.929764 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.929744 2578 scope.go:117] "RemoveContainer" containerID="d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94" Jul 15 21:31:57.938317 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.938298 2578 scope.go:117] "RemoveContainer" containerID="bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d" Jul 15 21:31:57.946558 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.946538 2578 scope.go:117] "RemoveContainer" containerID="0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e" Jul 15 21:31:57.946898 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:31:57.946877 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e\": container with ID starting with 0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e not found: ID does not exist" containerID="0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e" Jul 15 21:31:57.946988 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.946906 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e"} err="failed to get container status \"0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e\": rpc error: code = NotFound desc = could not find container \"0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e\": container with ID starting with 0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e not found: ID does not exist" Jul 15 21:31:57.946988 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.946955 2578 scope.go:117] "RemoveContainer" containerID="58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0" Jul 15 21:31:57.947255 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:31:57.947233 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0\": container with ID starting with 58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0 not found: ID does not exist" containerID="58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0" Jul 15 21:31:57.947379 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.947259 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0"} err="failed to get container status \"58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0\": rpc error: code = NotFound desc = could not find container \"58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0\": container with ID starting with 58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0 not found: ID does not exist" Jul 15 21:31:57.947379 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.947275 2578 scope.go:117] "RemoveContainer" containerID="1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1" Jul 15 21:31:57.947567 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:31:57.947545 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1\": container with ID starting with 1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1 not found: ID does not exist" containerID="1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1" Jul 15 21:31:57.947618 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.947578 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1"} err="failed to get container status \"1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1\": rpc error: code = NotFound desc = could not find container \"1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1\": container with ID starting with 1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1 not found: ID does not exist" Jul 15 21:31:57.947618 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.947600 2578 scope.go:117] "RemoveContainer" containerID="d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310" Jul 15 21:31:57.947865 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:31:57.947846 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310\": container with ID starting with d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310 not found: ID does not exist" containerID="d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310" Jul 15 21:31:57.947914 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.947871 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310"} err="failed to get container status \"d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310\": rpc error: code = NotFound desc = could not find container \"d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310\": container with ID starting with d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310 not found: ID does not exist" Jul 15 21:31:57.947914 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.947888 2578 scope.go:117] "RemoveContainer" containerID="6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556" Jul 15 21:31:57.948181 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:31:57.948163 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556\": container with ID starting with 6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556 not found: ID does not exist" containerID="6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556" Jul 15 21:31:57.948243 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.948186 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556"} err="failed to get container status \"6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556\": rpc error: code = NotFound desc = could not find container \"6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556\": container with ID starting with 6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556 not found: ID does not exist" Jul 15 21:31:57.948243 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.948201 2578 scope.go:117] "RemoveContainer" containerID="d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94" Jul 15 21:31:57.948408 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:31:57.948394 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94\": container with ID starting with d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94 not found: ID does not exist" containerID="d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94" Jul 15 21:31:57.948443 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.948410 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94"} err="failed to get container status \"d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94\": rpc error: code = NotFound desc = could not find container \"d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94\": container with ID starting with d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94 not found: ID does not exist" Jul 15 21:31:57.948443 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.948422 2578 scope.go:117] "RemoveContainer" containerID="bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d" Jul 15 21:31:57.948637 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:31:57.948622 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d\": container with ID starting with bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d not found: ID does not exist" containerID="bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d" Jul 15 21:31:57.948686 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.948641 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d"} err="failed to get container status \"bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d\": rpc error: code = NotFound desc = could not find container \"bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d\": container with ID starting with bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d not found: ID does not exist" Jul 15 21:31:57.948686 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.948654 2578 scope.go:117] "RemoveContainer" containerID="0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e" Jul 15 21:31:57.948859 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.948840 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e"} err="failed to get container status \"0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e\": rpc error: code = NotFound desc = could not find container \"0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e\": container with ID starting with 0c2074c520cb21e5cf1c916d84f25629e85fb3ce0d489bfbb3ad001b68f2562e not found: ID does not exist" Jul 15 21:31:57.948935 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.948860 2578 scope.go:117] "RemoveContainer" containerID="58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0" Jul 15 21:31:57.949072 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.949053 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0"} err="failed to get container status \"58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0\": rpc error: code = NotFound desc = could not find container \"58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0\": container with ID starting with 58d21bf42d84a3d3bdd827f25b95fd35f54d24278ece05cf8a46ef73de12b4a0 not found: ID does not exist" Jul 15 21:31:57.949133 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.949072 2578 scope.go:117] "RemoveContainer" containerID="1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1" Jul 15 21:31:57.949302 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.949286 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1"} err="failed to get container status \"1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1\": rpc error: code = NotFound desc = could not find container \"1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1\": container with ID starting with 1de26cfa52359860f300438da3ab106fe36d7b86c75dda6a98699960d2acf6e1 not found: ID does not exist" Jul 15 21:31:57.949334 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.949302 2578 scope.go:117] "RemoveContainer" containerID="d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310" Jul 15 21:31:57.949493 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.949476 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310"} err="failed to get container status \"d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310\": rpc error: code = NotFound desc = could not find container \"d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310\": container with ID starting with d99226de40f8747208c226f3581ff2ffc114631154ad94eae5e5b1ce60e41310 not found: ID does not exist" Jul 15 21:31:57.949531 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.949492 2578 scope.go:117] "RemoveContainer" containerID="6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556" Jul 15 21:31:57.949678 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.949662 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556"} err="failed to get container status \"6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556\": rpc error: code = NotFound desc = could not find container \"6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556\": container with ID starting with 6e3ad73f3504f3b40cdcf22797c9a8c96a94c502c9edb151c46831e1a95a0556 not found: ID does not exist" Jul 15 21:31:57.949772 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.949679 2578 scope.go:117] "RemoveContainer" containerID="d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94" Jul 15 21:31:57.949856 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.949841 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94"} err="failed to get container status \"d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94\": rpc error: code = NotFound desc = could not find container \"d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94\": container with ID starting with d3ad3499925155a4c3a9594418d37dfedfb0152754cb5d5c27d1a1520f85da94 not found: ID does not exist" Jul 15 21:31:57.949895 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.949855 2578 scope.go:117] "RemoveContainer" containerID="bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d" Jul 15 21:31:57.950024 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:57.950008 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d"} err="failed to get container status \"bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d\": rpc error: code = NotFound desc = could not find container \"bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d\": container with ID starting with bf3310832e164d817816e6674247f38fe3c9a69bce0b96b153295d6d7553773d not found: ID does not exist" Jul 15 21:31:59.049986 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:31:59.049948 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" path="/var/lib/kubelet/pods/b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7/volumes" Jul 15 21:32:00.503027 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.501534 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 21:32:00.503027 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.502060 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="prometheus" containerID="cri-o://204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f" gracePeriod=600 Jul 15 21:32:00.503027 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.502776 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2" gracePeriod=600 Jul 15 21:32:00.503027 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.502790 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="thanos-sidecar" containerID="cri-o://7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458" gracePeriod=600 Jul 15 21:32:00.503027 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.502903 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="config-reloader" containerID="cri-o://491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb" gracePeriod=600 Jul 15 21:32:00.503027 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.502990 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy" containerID="cri-o://96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a" gracePeriod=600 Jul 15 21:32:00.504897 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.502989 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy-web" containerID="cri-o://7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988" gracePeriod=600 Jul 15 21:32:00.749081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.749055 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:00.902426 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902393 2578 generic.go:356] "Generic (PLEG): container finished" podID="a699d543-58a5-4dc4-93d3-b52343415a86" containerID="d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2" exitCode=0 Jul 15 21:32:00.902426 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902420 2578 generic.go:356] "Generic (PLEG): container finished" podID="a699d543-58a5-4dc4-93d3-b52343415a86" containerID="96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a" exitCode=0 Jul 15 21:32:00.902426 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902427 2578 generic.go:356] "Generic (PLEG): container finished" podID="a699d543-58a5-4dc4-93d3-b52343415a86" containerID="7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988" exitCode=0 Jul 15 21:32:00.902426 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902432 2578 generic.go:356] "Generic (PLEG): container finished" podID="a699d543-58a5-4dc4-93d3-b52343415a86" containerID="7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458" exitCode=0 Jul 15 21:32:00.902426 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902437 2578 generic.go:356] "Generic (PLEG): container finished" podID="a699d543-58a5-4dc4-93d3-b52343415a86" containerID="491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb" exitCode=0 Jul 15 21:32:00.902819 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902443 2578 generic.go:356] "Generic (PLEG): container finished" podID="a699d543-58a5-4dc4-93d3-b52343415a86" containerID="204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f" exitCode=0 Jul 15 21:32:00.902819 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902473 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:00.902819 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902468 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerDied","Data":"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2"} Jul 15 21:32:00.902819 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902617 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerDied","Data":"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a"} Jul 15 21:32:00.902819 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902663 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerDied","Data":"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988"} Jul 15 21:32:00.902819 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902677 2578 scope.go:117] "RemoveContainer" containerID="d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2" Jul 15 21:32:00.902819 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902704 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerDied","Data":"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458"} Jul 15 21:32:00.902819 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902745 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerDied","Data":"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb"} Jul 15 21:32:00.902819 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902784 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerDied","Data":"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f"} Jul 15 21:32:00.902819 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.902822 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a699d543-58a5-4dc4-93d3-b52343415a86","Type":"ContainerDied","Data":"4bebc8735f97af265520854a1cfc6dd551484ccc625a85773f23120a96b3cb55"} Jul 15 21:32:00.903752 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.903735 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-serving-certs-ca-bundle\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.903830 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.903786 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.903939 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.903919 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-tls-assets\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904053 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.903970 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-db\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904053 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904004 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-trusted-ca-bundle\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904053 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904041 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-metrics-client-ca\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904248 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904077 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-web-config\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904248 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904143 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-tls\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904248 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904150 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:32:00.904248 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904169 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-metrics-client-certs\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904248 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904206 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-config-out\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904248 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904239 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-1\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904529 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904262 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-grpc-tls\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904529 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904283 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-0\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904529 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904316 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-thanos-prometheus-http-client-file\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904529 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904340 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhkkk\" (UniqueName: \"kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-kube-api-access-bhkkk\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904529 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904371 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-kubelet-serving-ca-bundle\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904529 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904395 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-kube-rbac-proxy\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904529 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904421 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-config\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904529 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904463 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-2\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904529 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904500 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a699d543-58a5-4dc4-93d3-b52343415a86\" (UID: \"a699d543-58a5-4dc4-93d3-b52343415a86\") " Jul 15 21:32:00.904989 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904690 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:32:00.904989 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904772 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:00.904989 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904791 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:00.904989 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.904855 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:32:00.905246 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.905170 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:32:00.905306 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.905249 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:32:00.905740 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.905711 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:32:00.906061 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.906033 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:32:00.907530 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.907501 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:32:00.907616 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.907581 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:32:00.907688 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.907645 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:32:00.908505 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.908476 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:32:00.908597 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.908549 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:32:00.909026 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.908999 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-config-out" (OuterVolumeSpecName: "config-out") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:32:00.909161 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.909140 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:32:00.909547 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.909522 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:32:00.910084 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.910054 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-kube-api-access-bhkkk" (OuterVolumeSpecName: "kube-api-access-bhkkk") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "kube-api-access-bhkkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:32:00.910176 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.910122 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:32:00.910225 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.910197 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-config" (OuterVolumeSpecName: "config") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:32:00.910305 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.910281 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:32:00.912388 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.912370 2578 scope.go:117] "RemoveContainer" containerID="96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a" Jul 15 21:32:00.919541 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.919517 2578 scope.go:117] "RemoveContainer" containerID="7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988" Jul 15 21:32:00.920705 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.920684 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-web-config" (OuterVolumeSpecName: "web-config") pod "a699d543-58a5-4dc4-93d3-b52343415a86" (UID: "a699d543-58a5-4dc4-93d3-b52343415a86"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:32:00.926528 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.926507 2578 scope.go:117] "RemoveContainer" containerID="7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458" Jul 15 21:32:00.933292 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.933272 2578 scope.go:117] "RemoveContainer" containerID="491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb" Jul 15 21:32:00.939792 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.939774 2578 scope.go:117] "RemoveContainer" containerID="204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f" Jul 15 21:32:00.947359 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.947341 2578 scope.go:117] "RemoveContainer" containerID="4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500" Jul 15 21:32:00.954107 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.954070 2578 scope.go:117] "RemoveContainer" containerID="d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2" Jul 15 21:32:00.954357 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:32:00.954339 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": container with ID starting with d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2 not found: ID does not exist" containerID="d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2" Jul 15 21:32:00.954409 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.954367 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2"} err="failed to get container status \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": rpc error: code = NotFound desc = could not find container \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": container with ID starting with d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2 not found: ID does not exist" Jul 15 21:32:00.954409 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.954386 2578 scope.go:117] "RemoveContainer" containerID="96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a" Jul 15 21:32:00.954610 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:32:00.954593 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": container with ID starting with 96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a not found: ID does not exist" containerID="96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a" Jul 15 21:32:00.954672 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.954617 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a"} err="failed to get container status \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": rpc error: code = NotFound desc = could not find container \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": container with ID starting with 96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a not found: ID does not exist" Jul 15 21:32:00.954672 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.954635 2578 scope.go:117] "RemoveContainer" containerID="7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988" Jul 15 21:32:00.954891 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:32:00.954856 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": container with ID starting with 7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988 not found: ID does not exist" containerID="7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988" Jul 15 21:32:00.954937 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.954895 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988"} err="failed to get container status \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": rpc error: code = NotFound desc = could not find container \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": container with ID starting with 7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988 not found: ID does not exist" Jul 15 21:32:00.954937 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.954910 2578 scope.go:117] "RemoveContainer" containerID="7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458" Jul 15 21:32:00.955174 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:32:00.955156 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": container with ID starting with 7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458 not found: ID does not exist" containerID="7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458" Jul 15 21:32:00.955221 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.955178 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458"} err="failed to get container status \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": rpc error: code = NotFound desc = could not find container \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": container with ID starting with 7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458 not found: ID does not exist" Jul 15 21:32:00.955221 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.955192 2578 scope.go:117] "RemoveContainer" containerID="491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb" Jul 15 21:32:00.955420 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:32:00.955403 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": container with ID starting with 491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb not found: ID does not exist" containerID="491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb" Jul 15 21:32:00.955472 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.955423 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb"} err="failed to get container status \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": rpc error: code = NotFound desc = could not find container \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": container with ID starting with 491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb not found: ID does not exist" Jul 15 21:32:00.955472 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.955436 2578 scope.go:117] "RemoveContainer" containerID="204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f" Jul 15 21:32:00.955674 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:32:00.955656 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": container with ID starting with 204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f not found: ID does not exist" containerID="204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f" Jul 15 21:32:00.955725 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.955677 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f"} err="failed to get container status \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": rpc error: code = NotFound desc = could not find container \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": container with ID starting with 204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f not found: ID does not exist" Jul 15 21:32:00.955725 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.955690 2578 scope.go:117] "RemoveContainer" containerID="4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500" Jul 15 21:32:00.955896 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:32:00.955882 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": container with ID starting with 4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500 not found: ID does not exist" containerID="4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500" Jul 15 21:32:00.955934 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.955898 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500"} err="failed to get container status \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": rpc error: code = NotFound desc = could not find container \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": container with ID starting with 4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500 not found: ID does not exist" Jul 15 21:32:00.955934 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.955910 2578 scope.go:117] "RemoveContainer" containerID="d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2" Jul 15 21:32:00.956217 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.956186 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2"} err="failed to get container status \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": rpc error: code = NotFound desc = could not find container \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": container with ID starting with d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2 not found: ID does not exist" Jul 15 21:32:00.956272 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.956217 2578 scope.go:117] "RemoveContainer" containerID="96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a" Jul 15 21:32:00.956419 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.956402 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a"} err="failed to get container status \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": rpc error: code = NotFound desc = could not find container \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": container with ID starting with 96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a not found: ID does not exist" Jul 15 21:32:00.956456 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.956419 2578 scope.go:117] "RemoveContainer" containerID="7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988" Jul 15 21:32:00.956625 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.956609 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988"} err="failed to get container status \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": rpc error: code = NotFound desc = could not find container \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": container with ID starting with 7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988 not found: ID does not exist" Jul 15 21:32:00.956625 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.956624 2578 scope.go:117] "RemoveContainer" containerID="7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458" Jul 15 21:32:00.956793 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.956778 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458"} err="failed to get container status \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": rpc error: code = NotFound desc = could not find container \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": container with ID starting with 7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458 not found: ID does not exist" Jul 15 21:32:00.956829 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.956792 2578 scope.go:117] "RemoveContainer" containerID="491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb" Jul 15 21:32:00.956947 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.956931 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb"} err="failed to get container status \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": rpc error: code = NotFound desc = could not find container \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": container with ID starting with 491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb not found: ID does not exist" Jul 15 21:32:00.956998 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.956946 2578 scope.go:117] "RemoveContainer" containerID="204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f" Jul 15 21:32:00.957170 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.957152 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f"} err="failed to get container status \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": rpc error: code = NotFound desc = could not find container \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": container with ID starting with 204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f not found: ID does not exist" Jul 15 21:32:00.957226 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.957169 2578 scope.go:117] "RemoveContainer" containerID="4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500" Jul 15 21:32:00.957346 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.957330 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500"} err="failed to get container status \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": rpc error: code = NotFound desc = could not find container \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": container with ID starting with 4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500 not found: ID does not exist" Jul 15 21:32:00.957383 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.957346 2578 scope.go:117] "RemoveContainer" containerID="d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2" Jul 15 21:32:00.957542 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.957523 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2"} err="failed to get container status \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": rpc error: code = NotFound desc = could not find container \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": container with ID starting with d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2 not found: ID does not exist" Jul 15 21:32:00.957579 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.957543 2578 scope.go:117] "RemoveContainer" containerID="96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a" Jul 15 21:32:00.957743 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.957728 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a"} err="failed to get container status \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": rpc error: code = NotFound desc = could not find container \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": container with ID starting with 96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a not found: ID does not exist" Jul 15 21:32:00.957801 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.957745 2578 scope.go:117] "RemoveContainer" containerID="7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988" Jul 15 21:32:00.957941 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.957924 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988"} err="failed to get container status \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": rpc error: code = NotFound desc = could not find container \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": container with ID starting with 7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988 not found: ID does not exist" Jul 15 21:32:00.957981 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.957941 2578 scope.go:117] "RemoveContainer" containerID="7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458" Jul 15 21:32:00.958157 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.958140 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458"} err="failed to get container status \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": rpc error: code = NotFound desc = could not find container \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": container with ID starting with 7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458 not found: ID does not exist" Jul 15 21:32:00.958203 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.958158 2578 scope.go:117] "RemoveContainer" containerID="491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb" Jul 15 21:32:00.958364 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.958346 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb"} err="failed to get container status \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": rpc error: code = NotFound desc = could not find container \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": container with ID starting with 491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb not found: ID does not exist" Jul 15 21:32:00.958406 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.958365 2578 scope.go:117] "RemoveContainer" containerID="204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f" Jul 15 21:32:00.958553 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.958538 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f"} err="failed to get container status \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": rpc error: code = NotFound desc = could not find container \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": container with ID starting with 204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f not found: ID does not exist" Jul 15 21:32:00.958591 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.958553 2578 scope.go:117] "RemoveContainer" containerID="4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500" Jul 15 21:32:00.958745 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.958730 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500"} err="failed to get container status \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": rpc error: code = NotFound desc = could not find container \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": container with ID starting with 4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500 not found: ID does not exist" Jul 15 21:32:00.958784 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.958745 2578 scope.go:117] "RemoveContainer" containerID="d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2" Jul 15 21:32:00.958968 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.958951 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2"} err="failed to get container status \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": rpc error: code = NotFound desc = could not find container \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": container with ID starting with d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2 not found: ID does not exist" Jul 15 21:32:00.958968 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.958966 2578 scope.go:117] "RemoveContainer" containerID="96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a" Jul 15 21:32:00.959185 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.959168 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a"} err="failed to get container status \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": rpc error: code = NotFound desc = could not find container \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": container with ID starting with 96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a not found: ID does not exist" Jul 15 21:32:00.959185 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.959185 2578 scope.go:117] "RemoveContainer" containerID="7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988" Jul 15 21:32:00.959389 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.959372 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988"} err="failed to get container status \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": rpc error: code = NotFound desc = could not find container \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": container with ID starting with 7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988 not found: ID does not exist" Jul 15 21:32:00.959437 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.959390 2578 scope.go:117] "RemoveContainer" containerID="7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458" Jul 15 21:32:00.959586 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.959565 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458"} err="failed to get container status \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": rpc error: code = NotFound desc = could not find container \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": container with ID starting with 7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458 not found: ID does not exist" Jul 15 21:32:00.959650 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.959589 2578 scope.go:117] "RemoveContainer" containerID="491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb" Jul 15 21:32:00.959793 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.959774 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb"} err="failed to get container status \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": rpc error: code = NotFound desc = could not find container \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": container with ID starting with 491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb not found: ID does not exist" Jul 15 21:32:00.959842 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.959793 2578 scope.go:117] "RemoveContainer" containerID="204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f" Jul 15 21:32:00.960018 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.960001 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f"} err="failed to get container status \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": rpc error: code = NotFound desc = could not find container \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": container with ID starting with 204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f not found: ID does not exist" Jul 15 21:32:00.960064 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.960019 2578 scope.go:117] "RemoveContainer" containerID="4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500" Jul 15 21:32:00.960246 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.960231 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500"} err="failed to get container status \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": rpc error: code = NotFound desc = could not find container \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": container with ID starting with 4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500 not found: ID does not exist" Jul 15 21:32:00.960287 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.960246 2578 scope.go:117] "RemoveContainer" containerID="d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2" Jul 15 21:32:00.960449 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.960433 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2"} err="failed to get container status \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": rpc error: code = NotFound desc = could not find container \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": container with ID starting with d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2 not found: ID does not exist" Jul 15 21:32:00.960489 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.960450 2578 scope.go:117] "RemoveContainer" containerID="96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a" Jul 15 21:32:00.960661 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.960644 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a"} err="failed to get container status \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": rpc error: code = NotFound desc = could not find container \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": container with ID starting with 96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a not found: ID does not exist" Jul 15 21:32:00.960661 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.960660 2578 scope.go:117] "RemoveContainer" containerID="7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988" Jul 15 21:32:00.960810 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.960796 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988"} err="failed to get container status \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": rpc error: code = NotFound desc = could not find container \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": container with ID starting with 7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988 not found: ID does not exist" Jul 15 21:32:00.960855 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.960809 2578 scope.go:117] "RemoveContainer" containerID="7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458" Jul 15 21:32:00.961003 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.960984 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458"} err="failed to get container status \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": rpc error: code = NotFound desc = could not find container \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": container with ID starting with 7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458 not found: ID does not exist" Jul 15 21:32:00.961051 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.961006 2578 scope.go:117] "RemoveContainer" containerID="491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb" Jul 15 21:32:00.961207 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.961193 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb"} err="failed to get container status \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": rpc error: code = NotFound desc = could not find container \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": container with ID starting with 491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb not found: ID does not exist" Jul 15 21:32:00.961253 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.961207 2578 scope.go:117] "RemoveContainer" containerID="204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f" Jul 15 21:32:00.961385 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.961365 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f"} err="failed to get container status \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": rpc error: code = NotFound desc = could not find container \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": container with ID starting with 204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f not found: ID does not exist" Jul 15 21:32:00.961420 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.961385 2578 scope.go:117] "RemoveContainer" containerID="4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500" Jul 15 21:32:00.961552 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.961539 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500"} err="failed to get container status \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": rpc error: code = NotFound desc = could not find container \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": container with ID starting with 4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500 not found: ID does not exist" Jul 15 21:32:00.961587 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.961552 2578 scope.go:117] "RemoveContainer" containerID="d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2" Jul 15 21:32:00.961731 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.961713 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2"} err="failed to get container status \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": rpc error: code = NotFound desc = could not find container \"d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2\": container with ID starting with d5a5ccfd633f9bdbf19a3f9c8eae22df6eb7414d8c4a9d65ae339e09f0d543f2 not found: ID does not exist" Jul 15 21:32:00.961771 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.961732 2578 scope.go:117] "RemoveContainer" containerID="96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a" Jul 15 21:32:00.961895 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.961880 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a"} err="failed to get container status \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": rpc error: code = NotFound desc = could not find container \"96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a\": container with ID starting with 96a71f6bbef35c13c96d80468c29e760e29d0f25a52f8463406d8ef8e8cb215a not found: ID does not exist" Jul 15 21:32:00.961895 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.961894 2578 scope.go:117] "RemoveContainer" containerID="7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988" Jul 15 21:32:00.962041 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.962026 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988"} err="failed to get container status \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": rpc error: code = NotFound desc = could not find container \"7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988\": container with ID starting with 7d2434d4d1c338fce090e0e83fc16764d6ff51cd9217bccf710cf2c49a132988 not found: ID does not exist" Jul 15 21:32:00.962105 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.962044 2578 scope.go:117] "RemoveContainer" containerID="7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458" Jul 15 21:32:00.962277 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.962256 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458"} err="failed to get container status \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": rpc error: code = NotFound desc = could not find container \"7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458\": container with ID starting with 7640bcfef791dff43e966c2525117dd12ab2caa8e95a9d7a0d0fc03a1dbd2458 not found: ID does not exist" Jul 15 21:32:00.962324 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.962277 2578 scope.go:117] "RemoveContainer" containerID="491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb" Jul 15 21:32:00.962507 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.962493 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb"} err="failed to get container status \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": rpc error: code = NotFound desc = could not find container \"491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb\": container with ID starting with 491198df48a6196e99f4e3e8c36b6847c2ffc0646ee95651238a7f510e8dc9fb not found: ID does not exist" Jul 15 21:32:00.962545 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.962507 2578 scope.go:117] "RemoveContainer" containerID="204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f" Jul 15 21:32:00.962696 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.962681 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f"} err="failed to get container status \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": rpc error: code = NotFound desc = could not find container \"204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f\": container with ID starting with 204c1693617002bd3fcfa85ef038dc6751080b25cd1775de5e77a83e2049487f not found: ID does not exist" Jul 15 21:32:00.962741 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.962696 2578 scope.go:117] "RemoveContainer" containerID="4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500" Jul 15 21:32:00.962900 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:00.962884 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500"} err="failed to get container status \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": rpc error: code = NotFound desc = could not find container \"4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500\": container with ID starting with 4054c59a702734ff9811a47b91b3cce1af24331ab4ddee70a7b0923bbceae500 not found: ID does not exist" Jul 15 21:32:01.005397 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005351 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005397 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005396 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-tls-assets\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005397 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005408 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-db\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005419 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005428 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-metrics-client-ca\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005438 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-web-config\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005446 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005453 2578 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-metrics-client-certs\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005461 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a699d543-58a5-4dc4-93d3-b52343415a86-config-out\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005469 2578 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-grpc-tls\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005477 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005484 2578 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005492 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhkkk\" (UniqueName: \"kubernetes.io/projected/a699d543-58a5-4dc4-93d3-b52343415a86-kube-api-access-bhkkk\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005501 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005509 2578 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-kube-rbac-proxy\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005517 2578 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-config\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005525 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a699d543-58a5-4dc4-93d3-b52343415a86-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.005581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.005535 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a699d543-58a5-4dc4-93d3-b52343415a86-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 21:32:01.220887 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.220782 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 21:32:01.228772 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.228745 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 21:32:01.252994 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.252963 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 21:32:01.253284 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253273 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="config-reloader" Jul 15 21:32:01.253323 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253286 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="config-reloader" Jul 15 21:32:01.253323 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253296 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="init-config-reloader" Jul 15 21:32:01.253323 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253301 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="init-config-reloader" Jul 15 21:32:01.253323 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253310 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy-thanos" Jul 15 21:32:01.253323 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253315 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy-thanos" Jul 15 21:32:01.253323 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253321 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="prom-label-proxy" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253327 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="prom-label-proxy" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253336 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253342 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253350 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="alertmanager" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253355 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="alertmanager" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253362 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253367 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253376 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99581639-234d-413f-be81-2ba54826a4cf" containerName="registry" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253381 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="99581639-234d-413f-be81-2ba54826a4cf" containerName="registry" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253387 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="init-config-reloader" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253392 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="init-config-reloader" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253399 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy-metric" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253406 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy-metric" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253415 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="thanos-sidecar" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253422 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="thanos-sidecar" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253428 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy-web" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253433 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy-web" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253440 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="config-reloader" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253445 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="config-reloader" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253455 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy-web" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253460 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy-web" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253465 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="prometheus" Jul 15 21:32:01.253497 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253470 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="prometheus" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253516 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy-metric" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253523 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="config-reloader" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253529 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="prom-label-proxy" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253535 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="config-reloader" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253540 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy-web" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253546 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="prometheus" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253552 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="99581639-234d-413f-be81-2ba54826a4cf" containerName="registry" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253556 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="kube-rbac-proxy" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253563 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1bc9c18-dcf0-456c-a003-8d8f21a4e7b7" containerName="alertmanager" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253569 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="thanos-sidecar" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253574 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253579 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy-thanos" Jul 15 21:32:01.254081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.253585 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" containerName="kube-rbac-proxy-web" Jul 15 21:32:01.257503 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.257482 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.260058 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260036 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 15 21:32:01.260058 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260053 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 15 21:32:01.260260 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260173 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 15 21:32:01.260424 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260408 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 15 21:32:01.260496 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260434 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 15 21:32:01.260496 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260482 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 15 21:32:01.260591 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260484 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 15 21:32:01.260812 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260676 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 15 21:32:01.260812 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260715 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 15 21:32:01.260812 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260764 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b3u7579tff5k\"" Jul 15 21:32:01.261074 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260821 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-l8sfg\"" Jul 15 21:32:01.261074 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.260998 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 15 21:32:01.261074 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.261002 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 15 21:32:01.261074 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.261056 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 15 21:32:01.264220 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.264190 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 15 21:32:01.265504 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.265488 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 15 21:32:01.275662 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.275640 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 21:32:01.409254 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409254 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409253 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409467 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409289 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409467 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409467 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409345 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409467 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409396 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-config\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409467 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409429 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409467 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409447 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409481 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409500 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409530 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409550 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409576 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhjzv\" (UniqueName: \"kubernetes.io/projected/5bfb3044-a331-4aa4-a4e2-22127ebef67c-kube-api-access-zhjzv\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409600 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409864 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409675 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5bfb3044-a331-4aa4-a4e2-22127ebef67c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409864 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409864 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409725 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5bfb3044-a331-4aa4-a4e2-22127ebef67c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.409864 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.409739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.510746 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.510746 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510693 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.510746 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-config\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.510746 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510746 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510785 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510801 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510816 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510850 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhjzv\" (UniqueName: \"kubernetes.io/projected/5bfb3044-a331-4aa4-a4e2-22127ebef67c-kube-api-access-zhjzv\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510940 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510965 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.510998 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5bfb3044-a331-4aa4-a4e2-22127ebef67c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.511023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.511050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5bfb3044-a331-4aa4-a4e2-22127ebef67c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.511074 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.511137 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.511168 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.511387 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.511208 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.512209 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.511688 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.512209 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.511869 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.513838 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.513761 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.514282 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.514082 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.514282 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.514164 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.514488 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.514424 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.514488 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.514465 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.515537 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.515254 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.515537 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.515332 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.515537 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.515382 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-config\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.515950 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.515648 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.515950 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.515667 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5bfb3044-a331-4aa4-a4e2-22127ebef67c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.515950 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.515847 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.516126 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.515963 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5bfb3044-a331-4aa4-a4e2-22127ebef67c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.516126 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.516035 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.516304 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.516284 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.516559 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.516543 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.517654 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.517637 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5bfb3044-a331-4aa4-a4e2-22127ebef67c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.518430 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.518413 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5bfb3044-a331-4aa4-a4e2-22127ebef67c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.522518 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.522495 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhjzv\" (UniqueName: \"kubernetes.io/projected/5bfb3044-a331-4aa4-a4e2-22127ebef67c-kube-api-access-zhjzv\") pod \"prometheus-k8s-0\" (UID: \"5bfb3044-a331-4aa4-a4e2-22127ebef67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.567248 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.567207 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:32:01.713607 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.713566 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 21:32:01.716383 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:32:01.716354 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfb3044_a331_4aa4_a4e2_22127ebef67c.slice/crio-65fb6fc778f594f5703d935fb8415602e72b9de0fb8c49d3482a051e02cc93f2 WatchSource:0}: Error finding container 65fb6fc778f594f5703d935fb8415602e72b9de0fb8c49d3482a051e02cc93f2: Status 404 returned error can't find the container with id 65fb6fc778f594f5703d935fb8415602e72b9de0fb8c49d3482a051e02cc93f2 Jul 15 21:32:01.913032 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.912998 2578 generic.go:356] "Generic (PLEG): container finished" podID="5bfb3044-a331-4aa4-a4e2-22127ebef67c" containerID="4737d0c0173b64bdf282210e4e5675b50240ae5e6711179dd77db2b3410144de" exitCode=0 Jul 15 21:32:01.913228 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.913104 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bfb3044-a331-4aa4-a4e2-22127ebef67c","Type":"ContainerDied","Data":"4737d0c0173b64bdf282210e4e5675b50240ae5e6711179dd77db2b3410144de"} Jul 15 21:32:01.913228 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:01.913217 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bfb3044-a331-4aa4-a4e2-22127ebef67c","Type":"ContainerStarted","Data":"65fb6fc778f594f5703d935fb8415602e72b9de0fb8c49d3482a051e02cc93f2"} Jul 15 21:32:02.922019 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:02.921983 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bfb3044-a331-4aa4-a4e2-22127ebef67c","Type":"ContainerStarted","Data":"46d29e2aee2fe0fd8c5c52be59d72786ebe987842b0b5c0da859c269a32e03bb"} Jul 15 21:32:02.922441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:02.922049 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bfb3044-a331-4aa4-a4e2-22127ebef67c","Type":"ContainerStarted","Data":"c6783459a24cca404f451187a283cd280882b37e686c899ca7806361e19029f7"} Jul 15 21:32:02.922441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:02.922074 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bfb3044-a331-4aa4-a4e2-22127ebef67c","Type":"ContainerStarted","Data":"f094f573e6a7d8060a6435f033a71a82ca83a5ac6c39e673c300bdd7fee56520"} Jul 15 21:32:02.922441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:02.922117 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bfb3044-a331-4aa4-a4e2-22127ebef67c","Type":"ContainerStarted","Data":"3c002019d0e36908b281c1318d85b08f5bb7a096df398c0a50d47fe6b26bb235"} Jul 15 21:32:02.922441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:02.922141 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bfb3044-a331-4aa4-a4e2-22127ebef67c","Type":"ContainerStarted","Data":"05b246bc1f6f7af9e1ee5d11542467571ae6c086b2c24bc45a414c05b35eb05a"} Jul 15 21:32:02.922441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:02.922162 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bfb3044-a331-4aa4-a4e2-22127ebef67c","Type":"ContainerStarted","Data":"29241aff8b49fc430ec5499ab58198baa7a3394816309535a76789689aaf976d"} Jul 15 21:32:02.949758 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:02.949694 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.930246221 podStartE2EDuration="1.949672167s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.425946ms" isStatefulPod=false podCreationTimestamp="2026-07-15 21:32:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:32:02.946983938 +0000 UTC m=+164.450419235" watchObservedRunningTime="2026-07-15 21:32:02.949672167 +0000 UTC m=+164.453107462" Jul 15 21:32:03.050916 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:03.050883 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a699d543-58a5-4dc4-93d3-b52343415a86" path="/var/lib/kubelet/pods/a699d543-58a5-4dc4-93d3-b52343415a86/volumes" Jul 15 21:32:06.568029 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:32:06.567982 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:33:01.567581 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:33:01.567540 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:33:01.583345 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:33:01.583312 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:33:02.185081 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:33:02.185051 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 21:34:18.939161 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:18.939131 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:34:18.942659 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:18.942635 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:34:18.942860 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:18.942735 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 21:34:48.724059 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.724020 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t"] Jul 15 21:34:48.727309 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.727293 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.729571 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.729548 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 21:34:48.730128 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.730048 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 21:34:48.730128 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.730099 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 21:34:48.730293 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.730099 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 21:34:48.730449 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.730435 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tt66b\"" Jul 15 21:34:48.741367 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.741332 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2d8688a8-e630-49a3-af86-c56bae64a404-manifests\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.741484 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.741396 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2d8688a8-e630-49a3-af86-c56bae64a404-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.741548 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.741477 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d8688a8-e630-49a3-af86-c56bae64a404-tmp\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.741548 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.741520 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d8688a8-e630-49a3-af86-c56bae64a404-webhook-cert\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.741646 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.741575 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq46f\" (UniqueName: \"kubernetes.io/projected/2d8688a8-e630-49a3-af86-c56bae64a404-kube-api-access-rq46f\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.742971 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.742947 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t"] Jul 15 21:34:48.842213 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.842165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d8688a8-e630-49a3-af86-c56bae64a404-tmp\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.842213 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.842216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d8688a8-e630-49a3-af86-c56bae64a404-webhook-cert\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.842439 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.842263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq46f\" (UniqueName: \"kubernetes.io/projected/2d8688a8-e630-49a3-af86-c56bae64a404-kube-api-access-rq46f\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.842439 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.842290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2d8688a8-e630-49a3-af86-c56bae64a404-manifests\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.842439 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.842316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2d8688a8-e630-49a3-af86-c56bae64a404-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.842654 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.842633 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2d8688a8-e630-49a3-af86-c56bae64a404-manifests\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.844544 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.844510 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d8688a8-e630-49a3-af86-c56bae64a404-tmp\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.844708 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.844690 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2d8688a8-e630-49a3-af86-c56bae64a404-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.844772 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.844745 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d8688a8-e630-49a3-af86-c56bae64a404-webhook-cert\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:48.850485 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:48.850462 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq46f\" (UniqueName: \"kubernetes.io/projected/2d8688a8-e630-49a3-af86-c56bae64a404-kube-api-access-rq46f\") pod \"opendatahub-operator-controller-manager-56fdfd95c4-c846t\" (UID: \"2d8688a8-e630-49a3-af86-c56bae64a404\") " pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:49.038146 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:49.038046 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:49.189867 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:49.189825 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t"] Jul 15 21:34:49.195046 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:34:49.195021 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d8688a8_e630_49a3_af86_c56bae64a404.slice/crio-71996285ccd261ab1d8170b1713d00630fd8ab4df32cd9b9f2464c1b530645ea WatchSource:0}: Error finding container 71996285ccd261ab1d8170b1713d00630fd8ab4df32cd9b9f2464c1b530645ea: Status 404 returned error can't find the container with id 71996285ccd261ab1d8170b1713d00630fd8ab4df32cd9b9f2464c1b530645ea Jul 15 21:34:49.196883 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:49.196865 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:34:49.603916 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:49.603880 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" event={"ID":"2d8688a8-e630-49a3-af86-c56bae64a404","Type":"ContainerStarted","Data":"71996285ccd261ab1d8170b1713d00630fd8ab4df32cd9b9f2464c1b530645ea"} Jul 15 21:34:52.624734 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:52.624644 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" event={"ID":"2d8688a8-e630-49a3-af86-c56bae64a404","Type":"ContainerStarted","Data":"4d19ea12d2b0a87c09f41c727c4f6c0551d584e1ef7088bdd89deebf2e745c6c"} Jul 15 21:34:53.630330 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:53.630295 2578 generic.go:356] "Generic (PLEG): container finished" podID="2d8688a8-e630-49a3-af86-c56bae64a404" containerID="4d19ea12d2b0a87c09f41c727c4f6c0551d584e1ef7088bdd89deebf2e745c6c" exitCode=0 Jul 15 21:34:53.630730 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:53.630383 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" event={"ID":"2d8688a8-e630-49a3-af86-c56bae64a404","Type":"ContainerDied","Data":"4d19ea12d2b0a87c09f41c727c4f6c0551d584e1ef7088bdd89deebf2e745c6c"} Jul 15 21:34:56.643711 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:56.643681 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" event={"ID":"2d8688a8-e630-49a3-af86-c56bae64a404","Type":"ContainerStarted","Data":"533dfe7c6713a9f2eaeb9a959156c2ee38c2ff88f23d8731e11cd9bb360e5a4b"} Jul 15 21:34:56.644065 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:56.643786 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:34:56.666325 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:34:56.666282 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" podStartSLOduration=2.423156691 podStartE2EDuration="8.666267694s" totalImagesPullingTime="5.178315319s" totalInitContainerRuntime="1.064795684s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:34:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:34:56.663982681 +0000 UTC m=+338.167417975" watchObservedRunningTime="2026-07-15 21:34:56.666267694 +0000 UTC m=+338.169702989" Jul 15 21:35:07.655185 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:07.655150 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56fdfd95c4-c846t" Jul 15 21:35:17.526070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.525980 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2"] Jul 15 21:35:17.529601 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.529578 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.532486 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.532463 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 21:35:17.533164 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.533125 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-hwnwx\"" Jul 15 21:35:17.533164 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.533138 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 21:35:17.533164 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.533158 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 21:35:17.540336 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.540315 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2"] Jul 15 21:35:17.592727 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.592697 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.592877 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.592737 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baa33ded-4297-4e9d-835a-79a36fc935f1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.592877 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.592769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.592877 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.592795 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.593011 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.592877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.593011 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.592903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.593011 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.592929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzgpp\" (UniqueName: \"kubernetes.io/projected/baa33ded-4297-4e9d-835a-79a36fc935f1-kube-api-access-wzgpp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.593150 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.593030 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.593150 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.593068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.694467 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.694424 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.694656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.694487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.694656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.694515 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.694656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.694544 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baa33ded-4297-4e9d-835a-79a36fc935f1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.694656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.694577 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.694656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.694605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.694656 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.694640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.694974 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.694665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.694974 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.694693 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzgpp\" (UniqueName: \"kubernetes.io/projected/baa33ded-4297-4e9d-835a-79a36fc935f1-kube-api-access-wzgpp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.695604 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.695539 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.695793 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.695710 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.695793 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.695752 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.697117 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.696015 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baa33ded-4297-4e9d-835a-79a36fc935f1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.697117 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.696684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.701439 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.698132 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.704391 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.704372 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.715880 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.715852 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baa33ded-4297-4e9d-835a-79a36fc935f1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.716135 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.716118 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzgpp\" (UniqueName: \"kubernetes.io/projected/baa33ded-4297-4e9d-835a-79a36fc935f1-kube-api-access-wzgpp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlcr2\" (UID: \"baa33ded-4297-4e9d-835a-79a36fc935f1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:17.841856 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:17.841824 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:18.003935 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:18.003866 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2"] Jul 15 21:35:18.006234 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:35:18.006206 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaa33ded_4297_4e9d_835a_79a36fc935f1.slice/crio-9a17e4fb91977f5a847cc307df40fbb16b608af8dfca62b9673f0633410257e1 WatchSource:0}: Error finding container 9a17e4fb91977f5a847cc307df40fbb16b608af8dfca62b9673f0633410257e1: Status 404 returned error can't find the container with id 9a17e4fb91977f5a847cc307df40fbb16b608af8dfca62b9673f0633410257e1 Jul 15 21:35:18.746150 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:18.746105 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" event={"ID":"baa33ded-4297-4e9d-835a-79a36fc935f1","Type":"ContainerStarted","Data":"9a17e4fb91977f5a847cc307df40fbb16b608af8dfca62b9673f0633410257e1"} Jul 15 21:35:20.239544 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.239509 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr"] Jul 15 21:35:20.248395 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.248366 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7"] Jul 15 21:35:20.248547 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.248463 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" Jul 15 21:35:20.250978 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.250954 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 15 21:35:20.251124 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.251012 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-frmqm\"" Jul 15 21:35:20.251857 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.251840 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:35:20.255317 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.255296 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8s6jx\"" Jul 15 21:35:20.255562 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.255543 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 15 21:35:20.257982 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.257960 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr"] Jul 15 21:35:20.262595 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.262014 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7"] Jul 15 21:35:20.317411 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.317372 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvbt5\" (UniqueName: \"kubernetes.io/projected/5e42f715-dd42-42d2-aec1-c7fdac6c186f-kube-api-access-mvbt5\") pod \"llmisvc-controller-manager-777c6bd7d5-2g8p7\" (UID: \"5e42f715-dd42-42d2-aec1-c7fdac6c186f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:35:20.317573 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.317481 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e42f715-dd42-42d2-aec1-c7fdac6c186f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2g8p7\" (UID: \"5e42f715-dd42-42d2-aec1-c7fdac6c186f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:35:20.317573 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.317513 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-679fl\" (UniqueName: \"kubernetes.io/projected/47e35d90-ed6f-4e87-9eec-970e6f30d3eb-kube-api-access-679fl\") pod \"kserve-controller-manager-d6f4b65b8-d5lcr\" (UID: \"47e35d90-ed6f-4e87-9eec-970e6f30d3eb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" Jul 15 21:35:20.317573 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.317541 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47e35d90-ed6f-4e87-9eec-970e6f30d3eb-cert\") pod \"kserve-controller-manager-d6f4b65b8-d5lcr\" (UID: \"47e35d90-ed6f-4e87-9eec-970e6f30d3eb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" Jul 15 21:35:20.329925 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.329897 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t"] Jul 15 21:35:20.333915 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.333896 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.336767 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.336743 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 21:35:20.336885 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.336794 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 21:35:20.336885 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.336812 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 21:35:20.336994 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.336743 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 21:35:20.337045 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.337022 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-sp7hq\"" Jul 15 21:35:20.337240 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.337225 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 21:35:20.342950 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.342917 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t"] Jul 15 21:35:20.418814 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.418755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9f3146b-acf3-4e25-af60-7658325c510e-cert\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.419009 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.418834 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e42f715-dd42-42d2-aec1-c7fdac6c186f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2g8p7\" (UID: \"5e42f715-dd42-42d2-aec1-c7fdac6c186f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:35:20.419009 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.418867 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679fl\" (UniqueName: \"kubernetes.io/projected/47e35d90-ed6f-4e87-9eec-970e6f30d3eb-kube-api-access-679fl\") pod \"kserve-controller-manager-d6f4b65b8-d5lcr\" (UID: \"47e35d90-ed6f-4e87-9eec-970e6f30d3eb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" Jul 15 21:35:20.419009 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.418906 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47e35d90-ed6f-4e87-9eec-970e6f30d3eb-cert\") pod \"kserve-controller-manager-d6f4b65b8-d5lcr\" (UID: \"47e35d90-ed6f-4e87-9eec-970e6f30d3eb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" Jul 15 21:35:20.419009 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.418944 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e9f3146b-acf3-4e25-af60-7658325c510e-metrics-cert\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.419009 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.418996 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvbt5\" (UniqueName: \"kubernetes.io/projected/5e42f715-dd42-42d2-aec1-c7fdac6c186f-kube-api-access-mvbt5\") pod \"llmisvc-controller-manager-777c6bd7d5-2g8p7\" (UID: \"5e42f715-dd42-42d2-aec1-c7fdac6c186f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:35:20.419288 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:35:20.419012 2578 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 15 21:35:20.419288 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.419030 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e9f3146b-acf3-4e25-af60-7658325c510e-manager-config\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.419288 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.419056 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdbm6\" (UniqueName: \"kubernetes.io/projected/e9f3146b-acf3-4e25-af60-7658325c510e-kube-api-access-xdbm6\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.419288 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:35:20.419081 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e42f715-dd42-42d2-aec1-c7fdac6c186f-cert podName:5e42f715-dd42-42d2-aec1-c7fdac6c186f nodeName:}" failed. No retries permitted until 2026-07-15 21:35:20.919059483 +0000 UTC m=+362.422494772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5e42f715-dd42-42d2-aec1-c7fdac6c186f-cert") pod "llmisvc-controller-manager-777c6bd7d5-2g8p7" (UID: "5e42f715-dd42-42d2-aec1-c7fdac6c186f") : secret "llmisvc-webhook-server-cert" not found Jul 15 21:35:20.419288 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:35:20.419160 2578 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 15 21:35:20.419288 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:35:20.419245 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47e35d90-ed6f-4e87-9eec-970e6f30d3eb-cert podName:47e35d90-ed6f-4e87-9eec-970e6f30d3eb nodeName:}" failed. No retries permitted until 2026-07-15 21:35:20.919224357 +0000 UTC m=+362.422659642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/47e35d90-ed6f-4e87-9eec-970e6f30d3eb-cert") pod "kserve-controller-manager-d6f4b65b8-d5lcr" (UID: "47e35d90-ed6f-4e87-9eec-970e6f30d3eb") : secret "kserve-webhook-server-cert" not found Jul 15 21:35:20.433186 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.432528 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvbt5\" (UniqueName: \"kubernetes.io/projected/5e42f715-dd42-42d2-aec1-c7fdac6c186f-kube-api-access-mvbt5\") pod \"llmisvc-controller-manager-777c6bd7d5-2g8p7\" (UID: \"5e42f715-dd42-42d2-aec1-c7fdac6c186f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:35:20.440223 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.440171 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-679fl\" (UniqueName: \"kubernetes.io/projected/47e35d90-ed6f-4e87-9eec-970e6f30d3eb-kube-api-access-679fl\") pod \"kserve-controller-manager-d6f4b65b8-d5lcr\" (UID: \"47e35d90-ed6f-4e87-9eec-970e6f30d3eb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" Jul 15 21:35:20.520057 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.519970 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e9f3146b-acf3-4e25-af60-7658325c510e-manager-config\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.520057 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.520017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdbm6\" (UniqueName: \"kubernetes.io/projected/e9f3146b-acf3-4e25-af60-7658325c510e-kube-api-access-xdbm6\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.520322 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.520072 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9f3146b-acf3-4e25-af60-7658325c510e-cert\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.520322 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.520182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e9f3146b-acf3-4e25-af60-7658325c510e-metrics-cert\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.520785 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.520759 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e9f3146b-acf3-4e25-af60-7658325c510e-manager-config\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.523151 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.523077 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9f3146b-acf3-4e25-af60-7658325c510e-cert\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.523849 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.523830 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e9f3146b-acf3-4e25-af60-7658325c510e-metrics-cert\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.537627 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.537599 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdbm6\" (UniqueName: \"kubernetes.io/projected/e9f3146b-acf3-4e25-af60-7658325c510e-kube-api-access-xdbm6\") pod \"lws-controller-manager-6577974b46-qhw5t\" (UID: \"e9f3146b-acf3-4e25-af60-7658325c510e\") " pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.645995 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.645949 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:20.664403 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.664354 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 21:35:20.664509 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.664460 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 21:35:20.664558 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.664514 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 21:35:20.766010 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.765966 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" event={"ID":"baa33ded-4297-4e9d-835a-79a36fc935f1","Type":"ContainerStarted","Data":"03214abdb4f2824e07c03ea09d71d3231d242b02a9614d2b93daebde1ffd3d5a"} Jul 15 21:35:20.788747 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.788610 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" podStartSLOduration=1.132790986 podStartE2EDuration="3.78858854s" totalImagesPullingTime="2.655797554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:35:20.787397681 +0000 UTC m=+362.290833011" watchObservedRunningTime="2026-07-15 21:35:20.78858854 +0000 UTC m=+362.292023835" Jul 15 21:35:20.810523 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.810493 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t"] Jul 15 21:35:20.812426 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:35:20.812399 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f3146b_acf3_4e25_af60_7658325c510e.slice/crio-8602624107da3d3b4676d6b46369f269918f2d842121f506193ec24d29e148be WatchSource:0}: Error finding container 8602624107da3d3b4676d6b46369f269918f2d842121f506193ec24d29e148be: Status 404 returned error can't find the container with id 8602624107da3d3b4676d6b46369f269918f2d842121f506193ec24d29e148be Jul 15 21:35:20.842191 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.842164 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:20.924496 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.924456 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47e35d90-ed6f-4e87-9eec-970e6f30d3eb-cert\") pod \"kserve-controller-manager-d6f4b65b8-d5lcr\" (UID: \"47e35d90-ed6f-4e87-9eec-970e6f30d3eb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" Jul 15 21:35:20.924685 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.924627 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e42f715-dd42-42d2-aec1-c7fdac6c186f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2g8p7\" (UID: \"5e42f715-dd42-42d2-aec1-c7fdac6c186f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:35:20.925061 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:35:20.925008 2578 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 15 21:35:20.925189 ip-10-0-130-62 kubenswrapper[2578]: E0715 21:35:20.925075 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e42f715-dd42-42d2-aec1-c7fdac6c186f-cert podName:5e42f715-dd42-42d2-aec1-c7fdac6c186f nodeName:}" failed. No retries permitted until 2026-07-15 21:35:21.925054349 +0000 UTC m=+363.428489631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5e42f715-dd42-42d2-aec1-c7fdac6c186f-cert") pod "llmisvc-controller-manager-777c6bd7d5-2g8p7" (UID: "5e42f715-dd42-42d2-aec1-c7fdac6c186f") : secret "llmisvc-webhook-server-cert" not found Jul 15 21:35:20.927510 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:20.927488 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47e35d90-ed6f-4e87-9eec-970e6f30d3eb-cert\") pod \"kserve-controller-manager-d6f4b65b8-d5lcr\" (UID: \"47e35d90-ed6f-4e87-9eec-970e6f30d3eb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" Jul 15 21:35:21.167108 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:21.167066 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" Jul 15 21:35:21.322608 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:21.322574 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr"] Jul 15 21:35:21.325977 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:35:21.325950 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e35d90_ed6f_4e87_9eec_970e6f30d3eb.slice/crio-d900b1ec18fe64482ccc29e3a34d3f8d8439530b49fea55f6f5256c3b3bd3c85 WatchSource:0}: Error finding container d900b1ec18fe64482ccc29e3a34d3f8d8439530b49fea55f6f5256c3b3bd3c85: Status 404 returned error can't find the container with id d900b1ec18fe64482ccc29e3a34d3f8d8439530b49fea55f6f5256c3b3bd3c85 Jul 15 21:35:21.774225 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:21.774158 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" event={"ID":"e9f3146b-acf3-4e25-af60-7658325c510e","Type":"ContainerStarted","Data":"8602624107da3d3b4676d6b46369f269918f2d842121f506193ec24d29e148be"} Jul 15 21:35:21.776070 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:21.776036 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" event={"ID":"47e35d90-ed6f-4e87-9eec-970e6f30d3eb","Type":"ContainerStarted","Data":"d900b1ec18fe64482ccc29e3a34d3f8d8439530b49fea55f6f5256c3b3bd3c85"} Jul 15 21:35:21.848295 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:21.848265 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:21.935745 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:21.935698 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e42f715-dd42-42d2-aec1-c7fdac6c186f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2g8p7\" (UID: \"5e42f715-dd42-42d2-aec1-c7fdac6c186f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:35:21.938570 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:21.938545 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e42f715-dd42-42d2-aec1-c7fdac6c186f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2g8p7\" (UID: \"5e42f715-dd42-42d2-aec1-c7fdac6c186f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:35:22.074257 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:22.074217 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:35:22.225320 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:22.225288 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7"] Jul 15 21:35:22.226944 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:35:22.226912 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5e42f715_dd42_42d2_aec1_c7fdac6c186f.slice/crio-b0d0d96a5b092347a1bac6e69762f093e435a7a9a6414fcea3f7d7fda93e49b3 WatchSource:0}: Error finding container b0d0d96a5b092347a1bac6e69762f093e435a7a9a6414fcea3f7d7fda93e49b3: Status 404 returned error can't find the container with id b0d0d96a5b092347a1bac6e69762f093e435a7a9a6414fcea3f7d7fda93e49b3 Jul 15 21:35:22.782252 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:22.782216 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" event={"ID":"5e42f715-dd42-42d2-aec1-c7fdac6c186f","Type":"ContainerStarted","Data":"b0d0d96a5b092347a1bac6e69762f093e435a7a9a6414fcea3f7d7fda93e49b3"} Jul 15 21:35:22.782724 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:22.782583 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:22.783603 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:22.783577 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlcr2" Jul 15 21:35:24.792750 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:24.792708 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" event={"ID":"47e35d90-ed6f-4e87-9eec-970e6f30d3eb","Type":"ContainerStarted","Data":"90bc668fb4070950d531056af588cff64866e80d613d1f35f71e9fd84ecb96b4"} Jul 15 21:35:24.809800 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:24.809754 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" podStartSLOduration=1.756775156 podStartE2EDuration="4.80974011s" totalImagesPullingTime="3.052964954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:35:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:35:24.808825977 +0000 UTC m=+366.312261294" watchObservedRunningTime="2026-07-15 21:35:24.80974011 +0000 UTC m=+366.313175405" Jul 15 21:35:25.798524 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:25.798481 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" event={"ID":"e9f3146b-acf3-4e25-af60-7658325c510e","Type":"ContainerStarted","Data":"57d5ed2b29f3e87ee968bb0d23246bcd2471b06d5b9ddd0296bc69dcd4b5a1c1"} Jul 15 21:35:25.798982 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:25.798623 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:25.798982 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:25.798662 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" Jul 15 21:35:25.816078 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:25.816030 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" podStartSLOduration=2.250136204 podStartE2EDuration="5.816014974s" totalImagesPullingTime="3.56587877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:35:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:35:25.815479225 +0000 UTC m=+367.318914521" watchObservedRunningTime="2026-07-15 21:35:25.816014974 +0000 UTC m=+367.319450266" Jul 15 21:35:27.809903 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:27.809865 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" event={"ID":"5e42f715-dd42-42d2-aec1-c7fdac6c186f","Type":"ContainerStarted","Data":"b22b37392c8b43e821b47cec170ffc4e31111fb1b791305b981f62ffc0e4af3b"} Jul 15 21:35:27.810290 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:27.809989 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:35:27.829246 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:27.829185 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" podStartSLOduration=2.760294779 podStartE2EDuration="7.829165082s" totalImagesPullingTime="5.068870303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:35:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:35:27.827547697 +0000 UTC m=+369.330983045" watchObservedRunningTime="2026-07-15 21:35:27.829165082 +0000 UTC m=+369.332600379" Jul 15 21:35:36.806430 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:36.806399 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6577974b46-qhw5t" Jul 15 21:35:56.809395 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:56.809363 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d5lcr" Jul 15 21:35:58.816307 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:35:58.816277 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2g8p7" Jul 15 21:36:50.160669 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:50.160621 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-glrgs"] Jul 15 21:36:50.164261 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:50.164232 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-glrgs" Jul 15 21:36:50.167713 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:50.167684 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8snnf\"" Jul 15 21:36:50.168576 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:50.168555 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 21:36:50.168699 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:50.168555 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 21:36:50.174294 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:50.174268 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-glrgs"] Jul 15 21:36:50.230774 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:50.230742 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6wkh\" (UniqueName: \"kubernetes.io/projected/43bf09ee-da9c-4d02-b54a-eb0cb25c235c-kube-api-access-j6wkh\") pod \"authorino-operator-75cc8d5cb7-glrgs\" (UID: \"43bf09ee-da9c-4d02-b54a-eb0cb25c235c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-glrgs" Jul 15 21:36:50.332035 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:50.331980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6wkh\" (UniqueName: \"kubernetes.io/projected/43bf09ee-da9c-4d02-b54a-eb0cb25c235c-kube-api-access-j6wkh\") pod \"authorino-operator-75cc8d5cb7-glrgs\" (UID: \"43bf09ee-da9c-4d02-b54a-eb0cb25c235c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-glrgs" Jul 15 21:36:50.340224 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:50.340191 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6wkh\" (UniqueName: \"kubernetes.io/projected/43bf09ee-da9c-4d02-b54a-eb0cb25c235c-kube-api-access-j6wkh\") pod \"authorino-operator-75cc8d5cb7-glrgs\" (UID: \"43bf09ee-da9c-4d02-b54a-eb0cb25c235c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-glrgs" Jul 15 21:36:50.476016 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:50.475924 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-glrgs" Jul 15 21:36:50.615274 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:50.615247 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-glrgs"] Jul 15 21:36:50.617991 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:36:50.617968 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43bf09ee_da9c_4d02_b54a_eb0cb25c235c.slice/crio-072826fa4390220653862224108daf081a99c1bda76e8fc8dbe1fd54499d7d10 WatchSource:0}: Error finding container 072826fa4390220653862224108daf081a99c1bda76e8fc8dbe1fd54499d7d10: Status 404 returned error can't find the container with id 072826fa4390220653862224108daf081a99c1bda76e8fc8dbe1fd54499d7d10 Jul 15 21:36:51.179516 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:51.179472 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-glrgs" event={"ID":"43bf09ee-da9c-4d02-b54a-eb0cb25c235c","Type":"ContainerStarted","Data":"072826fa4390220653862224108daf081a99c1bda76e8fc8dbe1fd54499d7d10"} Jul 15 21:36:54.195110 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:54.195064 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-glrgs" event={"ID":"43bf09ee-da9c-4d02-b54a-eb0cb25c235c","Type":"ContainerStarted","Data":"311f9e1895967503a92384b33fa48eb5dd9bc963c9ebf5dff2ce1d432aec978f"} Jul 15 21:36:54.195566 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:54.195167 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-glrgs" Jul 15 21:36:54.216849 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:36:54.216789 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-glrgs" podStartSLOduration=1.689426648 podStartE2EDuration="4.216773375s" totalImagesPullingTime="2.527346727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:36:54.214373198 +0000 UTC m=+455.717808492" watchObservedRunningTime="2026-07-15 21:36:54.216773375 +0000 UTC m=+455.720208737" Jul 15 21:37:05.202441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:05.202404 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-glrgs" Jul 15 21:37:13.820378 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:13.820341 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k"] Jul 15 21:37:13.824369 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:13.824349 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k" Jul 15 21:37:13.827200 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:13.827175 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5pcnt\"" Jul 15 21:37:13.833951 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:13.833924 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k"] Jul 15 21:37:13.852620 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:13.852586 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lttkm\" (UniqueName: \"kubernetes.io/projected/3501c000-4ed8-4bec-9f9d-901277947f9c-kube-api-access-lttkm\") pod \"limitador-operator-controller-manager-859d7b78bd-4v58k\" (UID: \"3501c000-4ed8-4bec-9f9d-901277947f9c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k" Jul 15 21:37:13.954085 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:13.954049 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttkm\" (UniqueName: \"kubernetes.io/projected/3501c000-4ed8-4bec-9f9d-901277947f9c-kube-api-access-lttkm\") pod \"limitador-operator-controller-manager-859d7b78bd-4v58k\" (UID: \"3501c000-4ed8-4bec-9f9d-901277947f9c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k" Jul 15 21:37:13.963599 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:13.963563 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lttkm\" (UniqueName: \"kubernetes.io/projected/3501c000-4ed8-4bec-9f9d-901277947f9c-kube-api-access-lttkm\") pod \"limitador-operator-controller-manager-859d7b78bd-4v58k\" (UID: \"3501c000-4ed8-4bec-9f9d-901277947f9c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k" Jul 15 21:37:14.136679 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:14.136580 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k" Jul 15 21:37:14.303488 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:14.303457 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k"] Jul 15 21:37:14.306466 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:37:14.306444 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3501c000_4ed8_4bec_9f9d_901277947f9c.slice/crio-662608a929fb60d8412288945c8951f59cba30e17234ddb2cfeb9873e387ec11 WatchSource:0}: Error finding container 662608a929fb60d8412288945c8951f59cba30e17234ddb2cfeb9873e387ec11: Status 404 returned error can't find the container with id 662608a929fb60d8412288945c8951f59cba30e17234ddb2cfeb9873e387ec11 Jul 15 21:37:15.294660 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:15.294627 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k" event={"ID":"3501c000-4ed8-4bec-9f9d-901277947f9c","Type":"ContainerStarted","Data":"662608a929fb60d8412288945c8951f59cba30e17234ddb2cfeb9873e387ec11"} Jul 15 21:37:17.307664 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:17.307629 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k" event={"ID":"3501c000-4ed8-4bec-9f9d-901277947f9c","Type":"ContainerStarted","Data":"cfcd2e0bfcc46a7daa0895fda3a3f7c6613fe8b188de1702f3eb62b639ed9a86"} Jul 15 21:37:17.308026 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:17.307686 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k" Jul 15 21:37:17.325844 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:17.325789 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k" podStartSLOduration=2.166017972 podStartE2EDuration="4.325774574s" totalImagesPullingTime="2.159756602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:37:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:37:17.324166032 +0000 UTC m=+478.827601327" watchObservedRunningTime="2026-07-15 21:37:17.325774574 +0000 UTC m=+478.829209868" Jul 15 21:37:28.315116 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:37:28.315071 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4v58k" Jul 15 21:39:18.969335 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:18.969308 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:39:18.971870 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:18.971847 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:39:41.667142 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.667042 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr"] Jul 15 21:39:41.670992 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.670968 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.674769 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.674745 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 21:39:41.674880 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.674814 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kpcw4\"" Jul 15 21:39:41.674958 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.674814 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 21:39:41.675014 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.674891 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 15 21:39:41.680596 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.680572 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr"] Jul 15 21:39:41.816496 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.816460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.816670 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.816526 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.816670 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.816569 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.816670 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.816584 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drnxs\" (UniqueName: \"kubernetes.io/projected/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-kube-api-access-drnxs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.816670 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.816610 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.816670 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.816647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.918220 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.918136 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.918220 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.918197 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.918458 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.918243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.918458 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.918311 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.918458 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.918360 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.918458 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.918386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drnxs\" (UniqueName: \"kubernetes.io/projected/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-kube-api-access-drnxs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.918765 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.918742 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.918835 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.918756 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.918835 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.918803 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.920674 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.920654 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.920771 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.920738 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.926212 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.926195 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drnxs\" (UniqueName: \"kubernetes.io/projected/252d9398-28ee-4c26-a78d-9b0c02b8c7fa-kube-api-access-drnxs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cffzr\" (UID: \"252d9398-28ee-4c26-a78d-9b0c02b8c7fa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:41.982609 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:41.982572 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:42.125200 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:42.116924 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr"] Jul 15 21:39:42.127028 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:39:42.126998 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod252d9398_28ee_4c26_a78d_9b0c02b8c7fa.slice/crio-2e8f9edeb5f03afcd4c65f3f1388623a80235a6605ffd748b9ddb9ff20374ebd WatchSource:0}: Error finding container 2e8f9edeb5f03afcd4c65f3f1388623a80235a6605ffd748b9ddb9ff20374ebd: Status 404 returned error can't find the container with id 2e8f9edeb5f03afcd4c65f3f1388623a80235a6605ffd748b9ddb9ff20374ebd Jul 15 21:39:42.996456 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:42.996414 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" event={"ID":"252d9398-28ee-4c26-a78d-9b0c02b8c7fa","Type":"ContainerStarted","Data":"2e8f9edeb5f03afcd4c65f3f1388623a80235a6605ffd748b9ddb9ff20374ebd"} Jul 15 21:39:44.002556 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:44.002516 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" event={"ID":"252d9398-28ee-4c26-a78d-9b0c02b8c7fa","Type":"ContainerStarted","Data":"a21749fc75936ce0b60752db93e0e0311244f105f9645ede8404db59a782420e"} Jul 15 21:39:44.021568 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:44.021508 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" podStartSLOduration=2.008461693 podStartE2EDuration="3.021491654s" totalImagesPullingTime="1.013029961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:39:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:39:44.020409985 +0000 UTC m=+625.523845283" watchObservedRunningTime="2026-07-15 21:39:44.021491654 +0000 UTC m=+625.524926948" Jul 15 21:39:51.983422 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:51.983386 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:51.985825 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:51.983462 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:51.996295 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:51.996270 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:52.050394 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:52.050365 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cffzr" Jul 15 21:39:53.981124 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:53.981077 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd"] Jul 15 21:39:53.987956 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:53.987931 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:53.990489 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:53.990468 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 15 21:39:53.994458 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:53.994438 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd"] Jul 15 21:39:54.136752 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.136720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.136954 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.136764 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.136954 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.136794 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btbpc\" (UniqueName: \"kubernetes.io/projected/25ee8458-22df-44d7-8e70-e3cba90f7b06-kube-api-access-btbpc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.136954 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.136823 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.136954 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.136904 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25ee8458-22df-44d7-8e70-e3cba90f7b06-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.136954 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.136938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.237557 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.237462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.237557 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.237507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btbpc\" (UniqueName: \"kubernetes.io/projected/25ee8458-22df-44d7-8e70-e3cba90f7b06-kube-api-access-btbpc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.237557 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.237535 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.237821 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.237581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25ee8458-22df-44d7-8e70-e3cba90f7b06-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.237821 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.237615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.237821 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.237720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.237997 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.237961 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.238055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.237997 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.238055 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.237967 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.239997 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.239962 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25ee8458-22df-44d7-8e70-e3cba90f7b06-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.240231 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.240024 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25ee8458-22df-44d7-8e70-e3cba90f7b06-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.245441 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.245418 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btbpc\" (UniqueName: \"kubernetes.io/projected/25ee8458-22df-44d7-8e70-e3cba90f7b06-kube-api-access-btbpc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd\" (UID: \"25ee8458-22df-44d7-8e70-e3cba90f7b06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.299961 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.299924 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:39:54.432529 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.432502 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd"] Jul 15 21:39:54.434510 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:39:54.434476 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25ee8458_22df_44d7_8e70_e3cba90f7b06.slice/crio-15f501d213903c35e83b5ad915dad6fdadf6df8afcbd39ab8baf3c1afdb2b738 WatchSource:0}: Error finding container 15f501d213903c35e83b5ad915dad6fdadf6df8afcbd39ab8baf3c1afdb2b738: Status 404 returned error can't find the container with id 15f501d213903c35e83b5ad915dad6fdadf6df8afcbd39ab8baf3c1afdb2b738 Jul 15 21:39:54.436028 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:54.436006 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:39:55.055617 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:55.055576 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" event={"ID":"25ee8458-22df-44d7-8e70-e3cba90f7b06","Type":"ContainerStarted","Data":"28bfcbd8e4900a330c94b0d42a36ff232d585956e177b75d913d9a6e2be634ab"} Jul 15 21:39:55.056003 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:55.055646 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" event={"ID":"25ee8458-22df-44d7-8e70-e3cba90f7b06","Type":"ContainerStarted","Data":"15f501d213903c35e83b5ad915dad6fdadf6df8afcbd39ab8baf3c1afdb2b738"} Jul 15 21:39:55.072135 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:39:55.072019 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" podStartSLOduration=1.883358684 podStartE2EDuration="2.072000732s" totalImagesPullingTime="188.642048ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:39:55.071233068 +0000 UTC m=+636.574668364" watchObservedRunningTime="2026-07-15 21:39:55.072000732 +0000 UTC m=+636.575436029" Jul 15 21:40:04.301057 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:04.301014 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:40:04.301482 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:04.301080 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:40:04.313659 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:04.313624 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:40:05.111938 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:05.111909 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd" Jul 15 21:40:11.292640 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.292600 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv"] Jul 15 21:40:11.296805 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.296782 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.299277 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.299258 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 15 21:40:11.303160 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.303137 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv"] Jul 15 21:40:11.396470 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.396437 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.396645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.396483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnqc8\" (UniqueName: \"kubernetes.io/projected/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-kube-api-access-dnqc8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.396645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.396528 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.396645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.396569 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.396645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.396607 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.396645 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.396630 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.497488 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.497452 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.497488 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.497498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnqc8\" (UniqueName: \"kubernetes.io/projected/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-kube-api-access-dnqc8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.497743 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.497526 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.497743 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.497571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.497743 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.497607 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.497743 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.497638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.497959 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.497933 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.498061 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.498038 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.498204 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.498108 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.499946 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.499925 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.500157 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.500140 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.504666 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.504642 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnqc8\" (UniqueName: \"kubernetes.io/projected/5ec204ab-7e7a-4d5c-a2a9-db54a115edc8-kube-api-access-dnqc8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv\" (UID: \"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.607679 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.607590 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:11.739380 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:11.739265 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv"] Jul 15 21:40:11.742197 ip-10-0-130-62 kubenswrapper[2578]: W0715 21:40:11.742166 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ec204ab_7e7a_4d5c_a2a9_db54a115edc8.slice/crio-b42bb211f97e24128aa8f2773747a599fe13c48ec48b34cd089480beb3684cf5 WatchSource:0}: Error finding container b42bb211f97e24128aa8f2773747a599fe13c48ec48b34cd089480beb3684cf5: Status 404 returned error can't find the container with id b42bb211f97e24128aa8f2773747a599fe13c48ec48b34cd089480beb3684cf5 Jul 15 21:40:12.135929 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:12.135892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" event={"ID":"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8","Type":"ContainerStarted","Data":"79357d5c18f2ab0e19cb1b78c74a56ec73f51384b1f8027162e0176e5e96c1af"} Jul 15 21:40:12.135929 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:12.135930 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" event={"ID":"5ec204ab-7e7a-4d5c-a2a9-db54a115edc8","Type":"ContainerStarted","Data":"b42bb211f97e24128aa8f2773747a599fe13c48ec48b34cd089480beb3684cf5"} Jul 15 21:40:12.152726 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:12.152676 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" podStartSLOduration=0.890413675 podStartE2EDuration="1.152660259s" totalImagesPullingTime="262.246584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:40:12.151940834 +0000 UTC m=+653.655376146" watchObservedRunningTime="2026-07-15 21:40:12.152660259 +0000 UTC m=+653.656095554" Jul 15 21:40:21.608664 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:21.608634 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:21.608664 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:21.608673 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:21.620758 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:21.620737 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:40:22.193406 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:40:22.193379 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv" Jul 15 21:44:19.005633 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:44:19.005562 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:44:19.006221 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:44:19.006199 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:49:19.035707 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:49:19.035670 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:49:19.037371 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:49:19.037350 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:54:19.064857 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:54:19.064826 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:54:19.068136 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:54:19.068115 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:59:19.093500 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:59:19.093466 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 21:59:19.098779 ip-10-0-130-62 kubenswrapper[2578]: I0715 21:59:19.098761 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 22:02:01.018015 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.017980 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp"] Jul 15 22:02:01.022036 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.022011 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.024818 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.024793 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a6de7e-openshift-default-dockercfg-rr7q4\"" Jul 15 22:02:01.037015 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.036990 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp"] Jul 15 22:02:01.132559 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.132519 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-podinfo\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.132821 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.132564 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-token\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.132821 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.132624 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istiod-ca-cert\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.132821 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.132718 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-data\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.132821 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.132764 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-credential-socket\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.132821 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.132792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-envoy\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.133004 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.132827 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-socket\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.133004 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.132923 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-certs\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.133004 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.132966 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6hkv\" (UniqueName: \"kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-kube-api-access-q6hkv\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.234288 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.234255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-podinfo\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.234288 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.234287 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-token\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.234529 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.234302 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istiod-ca-cert\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.234529 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.234345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-data\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.234529 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.234380 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-credential-socket\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.234529 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.234401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-envoy\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.234529 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.234434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-socket\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.234529 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.234491 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-certs\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.234816 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.234546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6hkv\" (UniqueName: \"kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-kube-api-access-q6hkv\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.234993 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.234965 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-certs\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.235144 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.235069 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-credential-socket\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.235144 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.235084 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-data\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.235266 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.235179 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istiod-ca-cert\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.235266 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.235253 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-socket\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.236913 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.236890 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-envoy\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.237019 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.236998 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-podinfo\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.244865 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.244842 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-token\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.245241 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.245220 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6hkv\" (UniqueName: \"kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-kube-api-access-q6hkv\") pod \"e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.335632 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.335586 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:01.472485 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.472458 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp"] Jul 15 22:02:01.474642 ip-10-0-130-62 kubenswrapper[2578]: W0715 22:02:01.474615 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f0603d3_5e72_4943_8cbc_f74ed41fa003.slice/crio-0dba0539b20ff4ae216957825a4590f9b4e9ecf7dc7608d221a5fe1f12cb0ca5 WatchSource:0}: Error finding container 0dba0539b20ff4ae216957825a4590f9b4e9ecf7dc7608d221a5fe1f12cb0ca5: Status 404 returned error can't find the container with id 0dba0539b20ff4ae216957825a4590f9b4e9ecf7dc7608d221a5fe1f12cb0ca5 Jul 15 22:02:01.476851 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.476818 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:02:01.476931 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.476880 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:02:01.476931 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:01.476910 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:02:02.324748 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:02.324713 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" event={"ID":"9f0603d3-5e72-4943-8cbc-f74ed41fa003","Type":"ContainerStarted","Data":"91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9"} Jul 15 22:02:02.325132 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:02.324769 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" event={"ID":"9f0603d3-5e72-4943-8cbc-f74ed41fa003","Type":"ContainerStarted","Data":"0dba0539b20ff4ae216957825a4590f9b4e9ecf7dc7608d221a5fe1f12cb0ca5"} Jul 15 22:02:02.336323 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:02.336299 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:02.340768 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:02.340745 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:02.344721 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:02.344670 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" podStartSLOduration=2.344653875 podStartE2EDuration="2.344653875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 22:02:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 22:02:02.342853475 +0000 UTC m=+1963.846288771" watchObservedRunningTime="2026-07-15 22:02:02.344653875 +0000 UTC m=+1963.848089240" Jul 15 22:02:03.329673 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:03.329630 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:03.330655 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:03.330636 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:19.372236 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:19.372200 2578 patch_prober.go:28] interesting pod/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 22:02:19.372614 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:19.372293 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 22:02:20.420799 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:20.420762 2578 generic.go:356] "Generic (PLEG): container finished" podID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerID="91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9" exitCode=0 Jul 15 22:02:20.421393 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:20.420838 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" event={"ID":"9f0603d3-5e72-4943-8cbc-f74ed41fa003","Type":"ContainerDied","Data":"91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9"} Jul 15 22:02:20.421393 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:20.421361 2578 scope.go:117] "RemoveContainer" containerID="91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9" Jul 15 22:02:20.422401 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:20.422374 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:02:20.422478 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:20.422465 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:02:20.422522 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:20.422502 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:02:21.427137 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:21.427070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" event={"ID":"9f0603d3-5e72-4943-8cbc-f74ed41fa003","Type":"ContainerStarted","Data":"62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534"} Jul 15 22:02:22.336761 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:22.336719 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:22.336977 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:22.336864 2578 patch_prober.go:28] interesting pod/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 15 22:02:22.336977 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:22.336915 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 15 22:02:23.336813 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:23.336771 2578 patch_prober.go:28] interesting pod/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 15 22:02:23.337306 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:23.336839 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 15 22:02:24.336850 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:24.336385 2578 patch_prober.go:28] interesting pod/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 15 22:02:24.336850 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:24.336458 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 15 22:02:25.340191 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:25.340156 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:25.340644 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:25.340422 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:02:25.341242 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:02:25.341227 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:03:55.300676 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:03:55.300637 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp"] Jul 15 22:03:55.301190 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:03:55.300832 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" containerID="cri-o://62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534" gracePeriod=30 Jul 15 22:03:55.342374 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:03:55.342341 2578 patch_prober.go:28] interesting pod/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 22:03:55.342527 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:03:55.342404 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 22:04:00.553940 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.553915 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:04:00.696654 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.696573 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-data\") pod \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " Jul 15 22:04:00.696654 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.696631 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6hkv\" (UniqueName: \"kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-kube-api-access-q6hkv\") pod \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " Jul 15 22:04:00.696949 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.696662 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-certs\") pod \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " Jul 15 22:04:00.696949 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.696682 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-podinfo\") pod \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " Jul 15 22:04:00.696949 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.696767 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-envoy\") pod \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " Jul 15 22:04:00.696949 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.696787 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-token\") pod \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " Jul 15 22:04:00.696949 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.696813 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-socket\") pod \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " Jul 15 22:04:00.696949 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.696841 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istiod-ca-cert\") pod \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " Jul 15 22:04:00.696949 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.696871 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-credential-socket\") pod \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\" (UID: \"9f0603d3-5e72-4943-8cbc-f74ed41fa003\") " Jul 15 22:04:00.697350 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.696984 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f0603d3-5e72-4943-8cbc-f74ed41fa003" (UID: "9f0603d3-5e72-4943-8cbc-f74ed41fa003"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:04:00.697350 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.697236 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f0603d3-5e72-4943-8cbc-f74ed41fa003" (UID: "9f0603d3-5e72-4943-8cbc-f74ed41fa003"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:04:00.697350 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.697302 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-certs\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:04:00.697508 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.697354 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f0603d3-5e72-4943-8cbc-f74ed41fa003" (UID: "9f0603d3-5e72-4943-8cbc-f74ed41fa003"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:04:00.697508 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.697431 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f0603d3-5e72-4943-8cbc-f74ed41fa003" (UID: "9f0603d3-5e72-4943-8cbc-f74ed41fa003"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:04:00.697616 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.697548 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f0603d3-5e72-4943-8cbc-f74ed41fa003" (UID: "9f0603d3-5e72-4943-8cbc-f74ed41fa003"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 22:04:00.699128 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.699075 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f0603d3-5e72-4943-8cbc-f74ed41fa003" (UID: "9f0603d3-5e72-4943-8cbc-f74ed41fa003"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 22:04:00.699253 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.699082 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f0603d3-5e72-4943-8cbc-f74ed41fa003" (UID: "9f0603d3-5e72-4943-8cbc-f74ed41fa003"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 22:04:00.699253 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.699211 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-kube-api-access-q6hkv" (OuterVolumeSpecName: "kube-api-access-q6hkv") pod "9f0603d3-5e72-4943-8cbc-f74ed41fa003" (UID: "9f0603d3-5e72-4943-8cbc-f74ed41fa003"). InnerVolumeSpecName "kube-api-access-q6hkv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 22:04:00.699332 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.699244 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f0603d3-5e72-4943-8cbc-f74ed41fa003" (UID: "9f0603d3-5e72-4943-8cbc-f74ed41fa003"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:04:00.798212 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.798180 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-data\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:04:00.798212 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.798206 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6hkv\" (UniqueName: \"kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-kube-api-access-q6hkv\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:04:00.798212 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.798215 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-podinfo\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:04:00.798212 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.798224 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-envoy\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:04:00.798475 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.798232 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istio-token\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:04:00.798475 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.798241 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-workload-socket\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:04:00.798475 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.798249 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f0603d3-5e72-4943-8cbc-f74ed41fa003-istiod-ca-cert\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:04:00.798475 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.798257 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0603d3-5e72-4943-8cbc-f74ed41fa003-credential-socket\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:04:00.925124 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.925077 2578 generic.go:356] "Generic (PLEG): container finished" podID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerID="62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534" exitCode=0 Jul 15 22:04:00.925319 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.925124 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" event={"ID":"9f0603d3-5e72-4943-8cbc-f74ed41fa003","Type":"ContainerDied","Data":"62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534"} Jul 15 22:04:00.925319 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.925161 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" Jul 15 22:04:00.925319 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.925180 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp" event={"ID":"9f0603d3-5e72-4943-8cbc-f74ed41fa003","Type":"ContainerDied","Data":"0dba0539b20ff4ae216957825a4590f9b4e9ecf7dc7608d221a5fe1f12cb0ca5"} Jul 15 22:04:00.925319 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.925201 2578 scope.go:117] "RemoveContainer" containerID="62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534" Jul 15 22:04:00.934898 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.934879 2578 scope.go:117] "RemoveContainer" containerID="91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9" Jul 15 22:04:00.942974 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.942957 2578 scope.go:117] "RemoveContainer" containerID="62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534" Jul 15 22:04:00.943249 ip-10-0-130-62 kubenswrapper[2578]: E0715 22:04:00.943232 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534\": container with ID starting with 62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534 not found: ID does not exist" containerID="62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534" Jul 15 22:04:00.943296 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.943258 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534"} err="failed to get container status \"62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534\": rpc error: code = NotFound desc = could not find container \"62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534\": container with ID starting with 62c7e39a108ffa2f52c315cd9c667710a1baf0365bac86ea8cad2733fc1a5534 not found: ID does not exist" Jul 15 22:04:00.943296 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.943276 2578 scope.go:117] "RemoveContainer" containerID="91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9" Jul 15 22:04:00.943519 ip-10-0-130-62 kubenswrapper[2578]: E0715 22:04:00.943503 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9\": container with ID starting with 91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9 not found: ID does not exist" containerID="91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9" Jul 15 22:04:00.943559 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.943526 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9"} err="failed to get container status \"91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9\": rpc error: code = NotFound desc = could not find container \"91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9\": container with ID starting with 91c1fe09b4e0ae24065c7533db5d6c22c7de1c21d475ed5fbd5a036f640a42b9 not found: ID does not exist" Jul 15 22:04:00.946948 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.946899 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp"] Jul 15 22:04:00.950741 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:00.950721 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a6de7e-openshift-default-6c4c9d8867-2w4cp"] Jul 15 22:04:01.051394 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:01.051361 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" path="/var/lib/kubelet/pods/9f0603d3-5e72-4943-8cbc-f74ed41fa003/volumes" Jul 15 22:04:04.798673 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.798636 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq"] Jul 15 22:04:04.799128 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.799084 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" Jul 15 22:04:04.799128 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.799125 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" Jul 15 22:04:04.799335 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.799136 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" Jul 15 22:04:04.799335 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.799145 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" Jul 15 22:04:04.799335 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.799275 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" Jul 15 22:04:04.799335 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.799291 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f0603d3-5e72-4943-8cbc-f74ed41fa003" containerName="istio-proxy" Jul 15 22:04:04.804296 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.804266 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.806837 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.806818 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ec9857-openshift-default-dockercfg-6zm4p\"" Jul 15 22:04:04.818455 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.818429 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq"] Jul 15 22:04:04.836852 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.836818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-certs\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.837007 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.836856 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istiod-ca-cert\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.837007 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.836888 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-podinfo\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.837007 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.836930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-data\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.837007 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.836962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-socket\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.837007 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.837001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-envoy\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.837223 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.837063 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhw8t\" (UniqueName: \"kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-kube-api-access-fhw8t\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.837223 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.837106 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-credential-socket\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.837223 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.837127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-token\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938137 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938076 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-certs\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938329 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istiod-ca-cert\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938329 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-podinfo\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938329 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938303 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-data\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938500 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-socket\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938500 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-envoy\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938500 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938428 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhw8t\" (UniqueName: \"kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-kube-api-access-fhw8t\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938500 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-credential-socket\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938500 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-certs\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938500 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938491 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-token\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938797 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938658 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-data\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938797 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-socket\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.938954 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.938932 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-credential-socket\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.939084 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.939051 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istiod-ca-cert\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.941153 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.941130 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-envoy\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.941406 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.941387 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-podinfo\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.945977 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.945951 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhw8t\" (UniqueName: \"kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-kube-api-access-fhw8t\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:04.946279 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:04.946263 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-token\") pod \"e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:05.114777 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:05.114684 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:05.253469 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:05.253443 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq"] Jul 15 22:04:05.255643 ip-10-0-130-62 kubenswrapper[2578]: W0715 22:04:05.255604 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a7ce10_0342_4570_905b_a5a5ac7a51cc.slice/crio-33d76e1ea5653e12f162dd66101ec5f83038338f665cbcb4dd51347f769a4d84 WatchSource:0}: Error finding container 33d76e1ea5653e12f162dd66101ec5f83038338f665cbcb4dd51347f769a4d84: Status 404 returned error can't find the container with id 33d76e1ea5653e12f162dd66101ec5f83038338f665cbcb4dd51347f769a4d84 Jul 15 22:04:05.257738 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:05.257707 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:04:05.257856 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:05.257770 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:04:05.257856 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:05.257799 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:04:05.952549 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:05.952516 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" event={"ID":"11a7ce10-0342-4570-905b-a5a5ac7a51cc","Type":"ContainerStarted","Data":"ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d"} Jul 15 22:04:05.952937 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:05.952568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" event={"ID":"11a7ce10-0342-4570-905b-a5a5ac7a51cc","Type":"ContainerStarted","Data":"33d76e1ea5653e12f162dd66101ec5f83038338f665cbcb4dd51347f769a4d84"} Jul 15 22:04:05.975214 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:05.975160 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" podStartSLOduration=1.975141504 podStartE2EDuration="1.975141504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 22:04:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 22:04:05.974047234 +0000 UTC m=+2087.477482529" watchObservedRunningTime="2026-07-15 22:04:05.975141504 +0000 UTC m=+2087.478576799" Jul 15 22:04:06.115786 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:06.115756 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:06.120320 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:06.120291 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:06.958402 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:06.958368 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:06.959319 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:06.959296 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:19.123357 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:19.123332 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 22:04:19.129840 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:19.129819 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 22:04:20.024051 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:20.024019 2578 generic.go:356] "Generic (PLEG): container finished" podID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerID="ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d" exitCode=0 Jul 15 22:04:20.024237 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:20.024107 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" event={"ID":"11a7ce10-0342-4570-905b-a5a5ac7a51cc","Type":"ContainerDied","Data":"ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d"} Jul 15 22:04:20.024545 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:20.024530 2578 scope.go:117] "RemoveContainer" containerID="ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d" Jul 15 22:04:20.025562 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:20.025535 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:04:20.025652 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:20.025603 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:04:20.025652 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:20.025633 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:04:21.031968 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:21.031919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" event={"ID":"11a7ce10-0342-4570-905b-a5a5ac7a51cc","Type":"ContainerStarted","Data":"b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd"} Jul 15 22:04:21.115112 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:21.115047 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:21.115975 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:21.115949 2578 patch_prober.go:28] interesting pod/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 15 22:04:21.116082 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:21.116007 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 15 22:04:21.959280 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:21.959233 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:22.115516 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:22.115486 2578 patch_prober.go:28] interesting pod/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 15 22:04:22.115893 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:22.115545 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 15 22:04:23.115813 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:23.115772 2578 patch_prober.go:28] interesting pod/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 15 22:04:23.116312 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:23.115845 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 15 22:04:24.115678 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:24.115642 2578 patch_prober.go:28] interesting pod/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 15 22:04:24.115849 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:24.115700 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 15 22:04:25.119419 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:25.119389 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:04:25.120243 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:04:25.120227 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:05:12.519062 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:12.518977 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq"] Jul 15 22:05:12.519637 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:12.519265 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerName="istio-proxy" containerID="cri-o://b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd" gracePeriod=30 Jul 15 22:05:17.780260 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.780202 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:05:17.857787 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.857763 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istiod-ca-cert\") pod \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " Jul 15 22:05:17.857926 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.857795 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-credential-socket\") pod \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " Jul 15 22:05:17.857926 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.857839 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhw8t\" (UniqueName: \"kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-kube-api-access-fhw8t\") pod \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " Jul 15 22:05:17.857926 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.857870 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-podinfo\") pod \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " Jul 15 22:05:17.857926 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.857904 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-envoy\") pod \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " Jul 15 22:05:17.857926 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.857919 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-data\") pod \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " Jul 15 22:05:17.858171 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.857933 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-token\") pod \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " Jul 15 22:05:17.858171 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.857961 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-socket\") pod \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " Jul 15 22:05:17.858171 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.857988 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-certs\") pod \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\" (UID: \"11a7ce10-0342-4570-905b-a5a5ac7a51cc\") " Jul 15 22:05:17.858171 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.858067 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11a7ce10-0342-4570-905b-a5a5ac7a51cc" (UID: "11a7ce10-0342-4570-905b-a5a5ac7a51cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:05:17.858369 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.858204 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11a7ce10-0342-4570-905b-a5a5ac7a51cc" (UID: "11a7ce10-0342-4570-905b-a5a5ac7a51cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 22:05:17.858369 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.858338 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11a7ce10-0342-4570-905b-a5a5ac7a51cc" (UID: "11a7ce10-0342-4570-905b-a5a5ac7a51cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:05:17.858473 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.858401 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11a7ce10-0342-4570-905b-a5a5ac7a51cc" (UID: "11a7ce10-0342-4570-905b-a5a5ac7a51cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:05:17.858473 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.858425 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istiod-ca-cert\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:05:17.858473 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.858445 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-credential-socket\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:05:17.858879 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.858853 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "11a7ce10-0342-4570-905b-a5a5ac7a51cc" (UID: "11a7ce10-0342-4570-905b-a5a5ac7a51cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:05:17.860280 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.860253 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11a7ce10-0342-4570-905b-a5a5ac7a51cc" (UID: "11a7ce10-0342-4570-905b-a5a5ac7a51cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 22:05:17.860366 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.860275 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "11a7ce10-0342-4570-905b-a5a5ac7a51cc" (UID: "11a7ce10-0342-4570-905b-a5a5ac7a51cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 22:05:17.860553 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.860530 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11a7ce10-0342-4570-905b-a5a5ac7a51cc" (UID: "11a7ce10-0342-4570-905b-a5a5ac7a51cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:05:17.860553 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.860534 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-kube-api-access-fhw8t" (OuterVolumeSpecName: "kube-api-access-fhw8t") pod "11a7ce10-0342-4570-905b-a5a5ac7a51cc" (UID: "11a7ce10-0342-4570-905b-a5a5ac7a51cc"). InnerVolumeSpecName "kube-api-access-fhw8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 22:05:17.959570 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.959540 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-envoy\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:05:17.959570 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.959562 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-data\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:05:17.959570 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.959570 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-token\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:05:17.959753 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.959579 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-socket\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:05:17.959753 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.959587 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a7ce10-0342-4570-905b-a5a5ac7a51cc-workload-certs\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:05:17.959753 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.959608 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhw8t\" (UniqueName: \"kubernetes.io/projected/11a7ce10-0342-4570-905b-a5a5ac7a51cc-kube-api-access-fhw8t\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:05:17.959753 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:17.959617 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11a7ce10-0342-4570-905b-a5a5ac7a51cc-istio-podinfo\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:05:18.328066 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.328033 2578 generic.go:356] "Generic (PLEG): container finished" podID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerID="b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd" exitCode=0 Jul 15 22:05:18.328266 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.328108 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" Jul 15 22:05:18.328266 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.328125 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" event={"ID":"11a7ce10-0342-4570-905b-a5a5ac7a51cc","Type":"ContainerDied","Data":"b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd"} Jul 15 22:05:18.328266 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.328183 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq" event={"ID":"11a7ce10-0342-4570-905b-a5a5ac7a51cc","Type":"ContainerDied","Data":"33d76e1ea5653e12f162dd66101ec5f83038338f665cbcb4dd51347f769a4d84"} Jul 15 22:05:18.328266 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.328205 2578 scope.go:117] "RemoveContainer" containerID="b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd" Jul 15 22:05:18.337450 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.337433 2578 scope.go:117] "RemoveContainer" containerID="ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d" Jul 15 22:05:18.344923 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.344897 2578 scope.go:117] "RemoveContainer" containerID="b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd" Jul 15 22:05:18.345230 ip-10-0-130-62 kubenswrapper[2578]: E0715 22:05:18.345211 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd\": container with ID starting with b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd not found: ID does not exist" containerID="b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd" Jul 15 22:05:18.345298 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.345237 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd"} err="failed to get container status \"b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd\": rpc error: code = NotFound desc = could not find container \"b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd\": container with ID starting with b0b0aca01f01cf18e60f06b5ef310888d70f44a8ba64caaf39a567ffd70847cd not found: ID does not exist" Jul 15 22:05:18.345298 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.345253 2578 scope.go:117] "RemoveContainer" containerID="ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d" Jul 15 22:05:18.345475 ip-10-0-130-62 kubenswrapper[2578]: E0715 22:05:18.345454 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d\": container with ID starting with ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d not found: ID does not exist" containerID="ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d" Jul 15 22:05:18.345530 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.345484 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d"} err="failed to get container status \"ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d\": rpc error: code = NotFound desc = could not find container \"ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d\": container with ID starting with ebb39d7d51617cec81b0ad2fbfa5075bd239decb186fdcff653e855dc7dfff7d not found: ID does not exist" Jul 15 22:05:18.349535 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.349512 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq"] Jul 15 22:05:18.354990 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:18.354970 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ec9857-openshift-default-db8fc4f5f-clrdq"] Jul 15 22:05:19.050960 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:19.050926 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" path="/var/lib/kubelet/pods/11a7ce10-0342-4570-905b-a5a5ac7a51cc/volumes" Jul 15 22:05:21.139365 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.139323 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69"] Jul 15 22:05:21.139895 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.139811 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerName="istio-proxy" Jul 15 22:05:21.139895 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.139826 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerName="istio-proxy" Jul 15 22:05:21.140012 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.139913 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerName="istio-proxy" Jul 15 22:05:21.140012 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.139928 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerName="istio-proxy" Jul 15 22:05:21.145280 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.145260 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.147827 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.147808 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-8cc1f1-openshift-default-dockercfg-cxgtb\"" Jul 15 22:05:21.152316 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.152282 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69"] Jul 15 22:05:21.188478 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.188446 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-token\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.188643 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.188503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-credential-socket\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.188643 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.188531 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-envoy\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.188643 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.188597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-certs\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.188777 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.188654 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-data\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.188777 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.188718 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-podinfo\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.188777 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.188744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80bab618-f687-4879-81b7-02f0bc91f8ba-istiod-ca-cert\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.188896 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.188819 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t48pn\" (UniqueName: \"kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-kube-api-access-t48pn\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.188896 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.188842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-socket\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.289730 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.289700 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-data\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.289917 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.289737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-podinfo\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.289917 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.289753 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80bab618-f687-4879-81b7-02f0bc91f8ba-istiod-ca-cert\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.289917 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.289803 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t48pn\" (UniqueName: \"kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-kube-api-access-t48pn\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.289917 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.289841 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-socket\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.289917 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.289866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-token\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.290200 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.290085 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-credential-socket\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.290200 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.290144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-envoy\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.290200 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.290167 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-data\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.290200 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.290190 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-certs\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.290400 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.290198 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-socket\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.290400 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.290251 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-credential-socket\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.291002 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.290981 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80bab618-f687-4879-81b7-02f0bc91f8ba-istiod-ca-cert\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.291512 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.291490 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-certs\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.292443 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.292421 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-envoy\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.293291 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.293274 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-podinfo\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.297299 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.297276 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-token\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.297457 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.297440 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t48pn\" (UniqueName: \"kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-kube-api-access-t48pn\") pod \"e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.457655 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.457552 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:21.593449 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.593418 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69"] Jul 15 22:05:21.597183 ip-10-0-130-62 kubenswrapper[2578]: W0715 22:05:21.597154 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80bab618_f687_4879_81b7_02f0bc91f8ba.slice/crio-9917bd038bba55c05fa6c38a708b34ed51406d0a094aa3adbb286a731b4fb8fa WatchSource:0}: Error finding container 9917bd038bba55c05fa6c38a708b34ed51406d0a094aa3adbb286a731b4fb8fa: Status 404 returned error can't find the container with id 9917bd038bba55c05fa6c38a708b34ed51406d0a094aa3adbb286a731b4fb8fa Jul 15 22:05:21.599709 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.599680 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:05:21.599803 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.599750 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:05:21.599803 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:21.599781 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:05:22.350665 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:22.350633 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" event={"ID":"80bab618-f687-4879-81b7-02f0bc91f8ba","Type":"ContainerStarted","Data":"8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d"} Jul 15 22:05:22.351014 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:22.350687 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" event={"ID":"80bab618-f687-4879-81b7-02f0bc91f8ba","Type":"ContainerStarted","Data":"9917bd038bba55c05fa6c38a708b34ed51406d0a094aa3adbb286a731b4fb8fa"} Jul 15 22:05:22.370913 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:22.370865 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" podStartSLOduration=1.3708508400000001 podStartE2EDuration="1.37085084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 22:05:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 22:05:22.369736389 +0000 UTC m=+2163.873171682" watchObservedRunningTime="2026-07-15 22:05:22.37085084 +0000 UTC m=+2163.874286134" Jul 15 22:05:22.457990 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:22.457959 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:22.463152 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:22.463110 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:23.363272 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:23.363231 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:23.364416 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:23.364389 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:38.364546 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:38.364512 2578 patch_prober.go:28] interesting pod/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 15 22:05:38.364955 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:38.364573 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 15 22:05:38.439218 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:38.439190 2578 generic.go:356] "Generic (PLEG): container finished" podID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerID="8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d" exitCode=0 Jul 15 22:05:38.439391 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:38.439271 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" event={"ID":"80bab618-f687-4879-81b7-02f0bc91f8ba","Type":"ContainerDied","Data":"8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d"} Jul 15 22:05:38.439733 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:38.439718 2578 scope.go:117] "RemoveContainer" containerID="8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d" Jul 15 22:05:38.440716 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:38.440690 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:05:38.440788 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:38.440754 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:05:38.440830 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:38.440803 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 22:05:39.446101 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:39.446051 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" event={"ID":"80bab618-f687-4879-81b7-02f0bc91f8ba","Type":"ContainerStarted","Data":"274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04"} Jul 15 22:05:39.458164 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:39.458143 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:39.458371 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:39.458348 2578 patch_prober.go:28] interesting pod/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 15 22:05:39.458414 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:39.458400 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 15 22:05:40.458012 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:40.457973 2578 patch_prober.go:28] interesting pod/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 15 22:05:40.458441 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:40.458047 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 15 22:05:41.458191 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:41.458161 2578 patch_prober.go:28] interesting pod/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 15 22:05:41.458569 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:41.458213 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 15 22:05:42.458902 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:42.458862 2578 patch_prober.go:28] interesting pod/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 15 22:05:42.459312 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:42.458921 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 15 22:05:43.458669 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:43.458631 2578 patch_prober.go:28] interesting pod/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 15 22:05:43.458892 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:43.458687 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 15 22:05:44.462140 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:44.462111 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:44.462606 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:44.462487 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:05:44.463116 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:05:44.463083 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:09:19.153502 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:09:19.153459 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 22:09:19.160403 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:09:19.160377 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hqhrr_09c29bbd-d242-45cc-b652-d1111a3f417e/ovn-acl-logging/0.log" Jul 15 22:11:35.838278 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:35.838189 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69"] Jul 15 22:11:35.838724 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:35.838451 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" containerID="cri-o://274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04" gracePeriod=30 Jul 15 22:11:41.094152 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.094125 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:11:41.202203 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.202082 2578 generic.go:356] "Generic (PLEG): container finished" podID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerID="274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04" exitCode=0 Jul 15 22:11:41.202203 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.202121 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" event={"ID":"80bab618-f687-4879-81b7-02f0bc91f8ba","Type":"ContainerDied","Data":"274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04"} Jul 15 22:11:41.202203 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.202163 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" Jul 15 22:11:41.202203 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.202195 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69" event={"ID":"80bab618-f687-4879-81b7-02f0bc91f8ba","Type":"ContainerDied","Data":"9917bd038bba55c05fa6c38a708b34ed51406d0a094aa3adbb286a731b4fb8fa"} Jul 15 22:11:41.202480 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.202228 2578 scope.go:117] "RemoveContainer" containerID="274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04" Jul 15 22:11:41.210721 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.210699 2578 scope.go:117] "RemoveContainer" containerID="8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d" Jul 15 22:11:41.218753 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.218730 2578 scope.go:117] "RemoveContainer" containerID="274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04" Jul 15 22:11:41.219032 ip-10-0-130-62 kubenswrapper[2578]: E0715 22:11:41.219012 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04\": container with ID starting with 274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04 not found: ID does not exist" containerID="274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04" Jul 15 22:11:41.219074 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.219043 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04"} err="failed to get container status \"274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04\": rpc error: code = NotFound desc = could not find container \"274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04\": container with ID starting with 274e91813d8976e4c38936da4877e43a67916a3b3b376e2e69a7a3ea21cb3f04 not found: ID does not exist" Jul 15 22:11:41.219074 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.219063 2578 scope.go:117] "RemoveContainer" containerID="8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d" Jul 15 22:11:41.219330 ip-10-0-130-62 kubenswrapper[2578]: E0715 22:11:41.219312 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d\": container with ID starting with 8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d not found: ID does not exist" containerID="8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d" Jul 15 22:11:41.219374 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.219338 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d"} err="failed to get container status \"8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d\": rpc error: code = NotFound desc = could not find container \"8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d\": container with ID starting with 8c458d7a335ce87b82cb79dbe87eeed8acb8f104d464a3f8a775694637fbb73d not found: ID does not exist" Jul 15 22:11:41.240406 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.240374 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-data\") pod \"80bab618-f687-4879-81b7-02f0bc91f8ba\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " Jul 15 22:11:41.240560 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.240490 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-certs\") pod \"80bab618-f687-4879-81b7-02f0bc91f8ba\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " Jul 15 22:11:41.240560 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.240517 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80bab618-f687-4879-81b7-02f0bc91f8ba-istiod-ca-cert\") pod \"80bab618-f687-4879-81b7-02f0bc91f8ba\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " Jul 15 22:11:41.240734 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.240562 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-token\") pod \"80bab618-f687-4879-81b7-02f0bc91f8ba\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " Jul 15 22:11:41.240734 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.240679 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t48pn\" (UniqueName: \"kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-kube-api-access-t48pn\") pod \"80bab618-f687-4879-81b7-02f0bc91f8ba\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " Jul 15 22:11:41.240734 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.240726 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-credential-socket\") pod \"80bab618-f687-4879-81b7-02f0bc91f8ba\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " Jul 15 22:11:41.240881 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.240755 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-envoy\") pod \"80bab618-f687-4879-81b7-02f0bc91f8ba\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " Jul 15 22:11:41.240881 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.240757 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "80bab618-f687-4879-81b7-02f0bc91f8ba" (UID: "80bab618-f687-4879-81b7-02f0bc91f8ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:11:41.240881 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.240785 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-podinfo\") pod \"80bab618-f687-4879-81b7-02f0bc91f8ba\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " Jul 15 22:11:41.240881 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.240811 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-socket\") pod \"80bab618-f687-4879-81b7-02f0bc91f8ba\" (UID: \"80bab618-f687-4879-81b7-02f0bc91f8ba\") " Jul 15 22:11:41.241058 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.240894 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80bab618-f687-4879-81b7-02f0bc91f8ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "80bab618-f687-4879-81b7-02f0bc91f8ba" (UID: "80bab618-f687-4879-81b7-02f0bc91f8ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 22:11:41.241274 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.241237 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "80bab618-f687-4879-81b7-02f0bc91f8ba" (UID: "80bab618-f687-4879-81b7-02f0bc91f8ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:11:41.241274 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.241255 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "80bab618-f687-4879-81b7-02f0bc91f8ba" (UID: "80bab618-f687-4879-81b7-02f0bc91f8ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:11:41.241274 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.241265 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-certs\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:11:41.241486 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.241292 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80bab618-f687-4879-81b7-02f0bc91f8ba-istiod-ca-cert\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:11:41.241486 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.241404 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "80bab618-f687-4879-81b7-02f0bc91f8ba" (UID: "80bab618-f687-4879-81b7-02f0bc91f8ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:11:41.242918 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.242882 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "80bab618-f687-4879-81b7-02f0bc91f8ba" (UID: "80bab618-f687-4879-81b7-02f0bc91f8ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 22:11:41.243043 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.242960 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "80bab618-f687-4879-81b7-02f0bc91f8ba" (UID: "80bab618-f687-4879-81b7-02f0bc91f8ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 22:11:41.243269 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.243242 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "80bab618-f687-4879-81b7-02f0bc91f8ba" (UID: "80bab618-f687-4879-81b7-02f0bc91f8ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 22:11:41.243269 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.243251 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-kube-api-access-t48pn" (OuterVolumeSpecName: "kube-api-access-t48pn") pod "80bab618-f687-4879-81b7-02f0bc91f8ba" (UID: "80bab618-f687-4879-81b7-02f0bc91f8ba"). InnerVolumeSpecName "kube-api-access-t48pn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 22:11:41.341967 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.341931 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-token\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:11:41.341967 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.341971 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t48pn\" (UniqueName: \"kubernetes.io/projected/80bab618-f687-4879-81b7-02f0bc91f8ba-kube-api-access-t48pn\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:11:41.342198 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.341981 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-credential-socket\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:11:41.342198 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.341990 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-envoy\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:11:41.342198 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.341998 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-podinfo\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:11:41.342198 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.342008 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-workload-socket\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:11:41.342198 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.342016 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80bab618-f687-4879-81b7-02f0bc91f8ba-istio-data\") on node \"ip-10-0-130-62.ec2.internal\" DevicePath \"\"" Jul 15 22:11:41.524121 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.524072 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69"] Jul 15 22:11:41.528140 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.528112 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-8cc1f1-openshift-default-dbcf86ff9-w4z69"] Jul 15 22:11:41.974617 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:41.974591 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-d5lcr_47e35d90-ed6f-4e87-9eec-970e6f30d3eb/manager/0.log" Jul 15 22:11:42.113961 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:42.113931 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-2g8p7_5e42f715-dd42-42d2-aec1-c7fdac6c186f/manager/0.log" Jul 15 22:11:42.901923 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:42.901892 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56fdfd95c4-c846t_2d8688a8-e630-49a3-af86-c56bae64a404/copy-manifests/0.log" Jul 15 22:11:42.908010 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:42.907980 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56fdfd95c4-c846t_2d8688a8-e630-49a3-af86-c56bae64a404/manager/0.log" Jul 15 22:11:43.051744 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:43.051710 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" path="/var/lib/kubelet/pods/80bab618-f687-4879-81b7-02f0bc91f8ba/volumes" Jul 15 22:11:46.170734 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:46.170702 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-glrgs_43bf09ee-da9c-4d02-b54a-eb0cb25c235c/manager/0.log" Jul 15 22:11:46.912668 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:46.912636 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4v58k_3501c000-4ed8-4bec-9f9d-901277947f9c/manager/0.log" Jul 15 22:11:47.278725 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:47.278635 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4hlcr2_baa33ded-4297-4e9d-835a-79a36fc935f1/istio-proxy/0.log" Jul 15 22:11:48.595243 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:48.595209 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-cffzr_252d9398-28ee-4c26-a78d-9b0c02b8c7fa/main/0.log" Jul 15 22:11:48.821810 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:48.821786 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmzkvd_25ee8458-22df-44d7-8e70-e3cba90f7b06/main/0.log" Jul 15 22:11:49.040980 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:49.040960 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-kg5fv_5ec204ab-7e7a-4d5c-a2a9-db54a115edc8/main/0.log" Jul 15 22:11:57.698143 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:57.698109 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6jqqs_52934511-13c5-4254-bdbd-b3e67a77c80f/global-pull-secret-syncer/0.log" Jul 15 22:11:57.857557 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:57.857525 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lbd68_d733c3ef-eaf2-44b5-8e84-c5f17c26665a/konnectivity-agent/0.log" Jul 15 22:11:57.941760 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:11:57.941728 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-62.ec2.internal_40d5eee52fbcaf2e5bbfd3cc3c45fbae/haproxy/0.log" Jul 15 22:12:03.551812 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:03.551755 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-glrgs_43bf09ee-da9c-4d02-b54a-eb0cb25c235c/manager/0.log" Jul 15 22:12:03.947244 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:03.947137 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4v58k_3501c000-4ed8-4bec-9f9d-901277947f9c/manager/0.log" Jul 15 22:12:05.778470 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:05.778437 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7647x_1c473fde-ec1e-436a-acf6-c3cf25bd25b7/kube-state-metrics/0.log" Jul 15 22:12:05.795601 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:05.795566 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7647x_1c473fde-ec1e-436a-acf6-c3cf25bd25b7/kube-rbac-proxy-main/0.log" Jul 15 22:12:05.815013 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:05.814972 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7647x_1c473fde-ec1e-436a-acf6-c3cf25bd25b7/kube-rbac-proxy-self/0.log" Jul 15 22:12:05.838305 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:05.838283 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-d7bb8877b-hfmt8_3bd4b704-c13e-41fc-b974-0abb041423da/metrics-server/0.log" Jul 15 22:12:05.863757 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:05.863730 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-5lkd4_97e3a6fb-d8e6-4979-91d7-a45c65aac66e/monitoring-plugin/0.log" Jul 15 22:12:06.018491 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.018462 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sggl4_71b48e14-1112-4ec7-a791-59a3bcb638fd/node-exporter/0.log" Jul 15 22:12:06.040863 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.040836 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sggl4_71b48e14-1112-4ec7-a791-59a3bcb638fd/kube-rbac-proxy/0.log" Jul 15 22:12:06.073599 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.073517 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sggl4_71b48e14-1112-4ec7-a791-59a3bcb638fd/init-textfile/0.log" Jul 15 22:12:06.099575 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.099549 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-ck4ng_e8ad9669-bb51-45d5-82dd-33a17a08a991/kube-rbac-proxy-main/0.log" Jul 15 22:12:06.118557 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.118530 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-ck4ng_e8ad9669-bb51-45d5-82dd-33a17a08a991/kube-rbac-proxy-self/0.log" Jul 15 22:12:06.138300 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.138274 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-ck4ng_e8ad9669-bb51-45d5-82dd-33a17a08a991/openshift-state-metrics/0.log" Jul 15 22:12:06.173766 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.173739 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bfb3044-a331-4aa4-a4e2-22127ebef67c/prometheus/0.log" Jul 15 22:12:06.191073 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.191052 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bfb3044-a331-4aa4-a4e2-22127ebef67c/config-reloader/0.log" Jul 15 22:12:06.210343 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.210313 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bfb3044-a331-4aa4-a4e2-22127ebef67c/thanos-sidecar/0.log" Jul 15 22:12:06.239372 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.239345 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bfb3044-a331-4aa4-a4e2-22127ebef67c/kube-rbac-proxy-web/0.log" Jul 15 22:12:06.258960 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.258935 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bfb3044-a331-4aa4-a4e2-22127ebef67c/kube-rbac-proxy/0.log" Jul 15 22:12:06.277805 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.277782 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bfb3044-a331-4aa4-a4e2-22127ebef67c/kube-rbac-proxy-thanos/0.log" Jul 15 22:12:06.300982 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.300963 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bfb3044-a331-4aa4-a4e2-22127ebef67c/init-config-reloader/0.log" Jul 15 22:12:06.362571 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.362495 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-rjhbm_086a628f-3fb3-4dce-b92c-b6ba7b203703/prometheus-operator-admission-webhook/0.log" Jul 15 22:12:06.460706 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.455059 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c54bc95ff-ll9hs_3eb8d285-7470-48fe-bbf0-2d52824af9bf/thanos-query/0.log" Jul 15 22:12:06.475504 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.475480 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c54bc95ff-ll9hs_3eb8d285-7470-48fe-bbf0-2d52824af9bf/kube-rbac-proxy-web/0.log" Jul 15 22:12:06.493923 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.493892 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c54bc95ff-ll9hs_3eb8d285-7470-48fe-bbf0-2d52824af9bf/kube-rbac-proxy/0.log" Jul 15 22:12:06.512472 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.512443 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c54bc95ff-ll9hs_3eb8d285-7470-48fe-bbf0-2d52824af9bf/prom-label-proxy/0.log" Jul 15 22:12:06.530407 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.530384 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c54bc95ff-ll9hs_3eb8d285-7470-48fe-bbf0-2d52824af9bf/kube-rbac-proxy-rules/0.log" Jul 15 22:12:06.549437 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:06.549413 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c54bc95ff-ll9hs_3eb8d285-7470-48fe-bbf0-2d52824af9bf/kube-rbac-proxy-metrics/0.log" Jul 15 22:12:07.684203 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.684177 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-npn72_0ae8a9c8-1f23-41d7-8983-7133759435e6/networking-console-plugin/0.log" Jul 15 22:12:07.820240 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.820204 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk"] Jul 15 22:12:07.820782 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.820753 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerName="istio-proxy" Jul 15 22:12:07.820782 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.820779 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="11a7ce10-0342-4570-905b-a5a5ac7a51cc" containerName="istio-proxy" Jul 15 22:12:07.820957 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.820798 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" Jul 15 22:12:07.820957 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.820807 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" Jul 15 22:12:07.820957 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.820819 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" Jul 15 22:12:07.820957 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.820828 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" Jul 15 22:12:07.820957 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.820939 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="80bab618-f687-4879-81b7-02f0bc91f8ba" containerName="istio-proxy" Jul 15 22:12:07.824254 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.824236 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.826947 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.826926 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rh4xt\"/\"kube-root-ca.crt\"" Jul 15 22:12:07.827114 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.826924 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rh4xt\"/\"openshift-service-ca.crt\"" Jul 15 22:12:07.827211 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.827041 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rh4xt\"/\"default-dockercfg-9jmjh\"" Jul 15 22:12:07.832901 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.832872 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk"] Jul 15 22:12:07.890389 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.890354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-sys\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.890389 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.890391 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-lib-modules\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.890629 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.890409 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkkcc\" (UniqueName: \"kubernetes.io/projected/fa2ff95a-e674-45db-8d2a-745d640a228e-kube-api-access-mkkcc\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.890629 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.890510 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-podres\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.890629 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.890562 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-proc\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.991981 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.991891 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-sys\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.991981 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.991928 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-lib-modules\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.991981 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.991947 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkkcc\" (UniqueName: \"kubernetes.io/projected/fa2ff95a-e674-45db-8d2a-745d640a228e-kube-api-access-mkkcc\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.992277 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.992019 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-sys\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.992277 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.992076 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-podres\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.992277 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.992131 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-lib-modules\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.992277 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.992153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-proc\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.992277 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.992237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-podres\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.992277 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.992251 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fa2ff95a-e674-45db-8d2a-745d640a228e-proc\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:07.999712 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:07.999684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkkcc\" (UniqueName: \"kubernetes.io/projected/fa2ff95a-e674-45db-8d2a-745d640a228e-kube-api-access-mkkcc\") pod \"perf-node-gather-daemonset-x8cxk\" (UID: \"fa2ff95a-e674-45db-8d2a-745d640a228e\") " pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:08.134356 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:08.134320 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:08.267555 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:08.267530 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk"] Jul 15 22:12:08.270105 ip-10-0-130-62 kubenswrapper[2578]: W0715 22:12:08.270064 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfa2ff95a_e674_45db_8d2a_745d640a228e.slice/crio-3977101e7a9fcab05a1a485dd3d63d45f5c6465ed9f798bc25ce8c76bab10d96 WatchSource:0}: Error finding container 3977101e7a9fcab05a1a485dd3d63d45f5c6465ed9f798bc25ce8c76bab10d96: Status 404 returned error can't find the container with id 3977101e7a9fcab05a1a485dd3d63d45f5c6465ed9f798bc25ce8c76bab10d96 Jul 15 22:12:08.343416 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:08.343385 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" event={"ID":"fa2ff95a-e674-45db-8d2a-745d640a228e","Type":"ContainerStarted","Data":"3977101e7a9fcab05a1a485dd3d63d45f5c6465ed9f798bc25ce8c76bab10d96"} Jul 15 22:12:08.855363 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:08.855335 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-z6vvf_ef049efd-fdaa-48eb-9766-9c4647e076c0/download-server/0.log" Jul 15 22:12:09.349364 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:09.349333 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" event={"ID":"fa2ff95a-e674-45db-8d2a-745d640a228e","Type":"ContainerStarted","Data":"bf4b5b297620ae16ebcbb7389fdae758206ce6333cc9d90cc71da12af4561815"} Jul 15 22:12:09.349550 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:09.349443 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:09.365622 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:09.365569 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" podStartSLOduration=2.365554046 podStartE2EDuration="2.365554046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 22:12:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 22:12:09.363400399 +0000 UTC m=+2570.866835696" watchObservedRunningTime="2026-07-15 22:12:09.365554046 +0000 UTC m=+2570.868989340" Jul 15 22:12:10.410584 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:10.410554 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ml6fl_07138502-cc97-44da-9b9f-a0ea92aa4e68/dns/0.log" Jul 15 22:12:10.429898 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:10.429874 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ml6fl_07138502-cc97-44da-9b9f-a0ea92aa4e68/kube-rbac-proxy/0.log" Jul 15 22:12:10.488646 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:10.488609 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nbzd8_4d05dda3-b68c-4486-9369-a0615903e606/dns-node-resolver/0.log" Jul 15 22:12:11.014179 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:11.014147 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-59d7c4c88f-z5dc7_fbc3b38c-86f5-49d4-a0b8-a4616817759b/registry/0.log" Jul 15 22:12:11.031125 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:11.031085 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-822j5_f191e06a-8191-4762-8e60-52d29e065cb2/node-ca/0.log" Jul 15 22:12:11.956759 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:11.956729 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4hlcr2_baa33ded-4297-4e9d-835a-79a36fc935f1/istio-proxy/0.log" Jul 15 22:12:13.049391 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:13.049353 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p7x5z_50607b0d-4426-4d4b-97af-4db2e1e305fa/serve-healthcheck-canary/0.log" Jul 15 22:12:13.505297 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:13.505216 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5rrtl_0cfdbf87-675b-4f5d-bccf-b57a4f27eade/kube-rbac-proxy/0.log" Jul 15 22:12:13.524956 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:13.524930 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5rrtl_0cfdbf87-675b-4f5d-bccf-b57a4f27eade/exporter/0.log" Jul 15 22:12:13.548471 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:13.548446 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5rrtl_0cfdbf87-675b-4f5d-bccf-b57a4f27eade/extractor/0.log" Jul 15 22:12:15.364081 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:15.364053 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rh4xt/perf-node-gather-daemonset-x8cxk" Jul 15 22:12:15.781977 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:15.781893 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-d5lcr_47e35d90-ed6f-4e87-9eec-970e6f30d3eb/manager/0.log" Jul 15 22:12:15.899392 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:15.899357 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-2g8p7_5e42f715-dd42-42d2-aec1-c7fdac6c186f/manager/0.log" Jul 15 22:12:16.376674 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:16.376634 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56fdfd95c4-c846t_2d8688a8-e630-49a3-af86-c56bae64a404/manager/0.log" Jul 15 22:12:16.395247 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:16.395224 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56fdfd95c4-c846t_2d8688a8-e630-49a3-af86-c56bae64a404/copy-manifests/0.log" Jul 15 22:12:17.854040 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:17.854010 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6577974b46-qhw5t_e9f3146b-acf3-4e25-af60-7658325c510e/manager/0.log" Jul 15 22:12:18.476146 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:18.476120 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4hlcr2_baa33ded-4297-4e9d-835a-79a36fc935f1/istio-proxy/0.log" Jul 15 22:12:20.020672 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:20.020639 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4hlcr2_baa33ded-4297-4e9d-835a-79a36fc935f1/istio-proxy/0.log" Jul 15 22:12:22.677886 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:22.677858 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-4rt8v_c27275b5-8cfc-4080-b948-0faf61ae132c/migrator/0.log" Jul 15 22:12:22.694837 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:22.694804 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-4rt8v_c27275b5-8cfc-4080-b948-0faf61ae132c/graceful-termination/0.log" Jul 15 22:12:24.122755 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:24.122730 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfpjx_dfb01554-4420-4174-a15a-1b67f0dfd799/kube-multus-additional-cni-plugins/0.log" Jul 15 22:12:24.141010 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:24.140983 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfpjx_dfb01554-4420-4174-a15a-1b67f0dfd799/egress-router-binary-copy/0.log" Jul 15 22:12:24.158390 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:24.158364 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfpjx_dfb01554-4420-4174-a15a-1b67f0dfd799/cni-plugins/0.log" Jul 15 22:12:24.239115 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:24.239064 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfpjx_dfb01554-4420-4174-a15a-1b67f0dfd799/bond-cni-plugin/0.log" Jul 15 22:12:24.257700 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:24.257668 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfpjx_dfb01554-4420-4174-a15a-1b67f0dfd799/routeoverride-cni/0.log" Jul 15 22:12:24.279172 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:24.279139 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfpjx_dfb01554-4420-4174-a15a-1b67f0dfd799/whereabouts-cni-bincopy/0.log" Jul 15 22:12:24.297260 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:24.297229 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfpjx_dfb01554-4420-4174-a15a-1b67f0dfd799/whereabouts-cni/0.log" Jul 15 22:12:24.507938 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:24.507862 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lbm84_ced8f5f2-4875-4a56-bd5d-03ac678d70ed/kube-multus/0.log" Jul 15 22:12:24.611538 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:24.611513 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j7s97_50eee8aa-8304-424c-8525-91b270d118ac/network-metrics-daemon/0.log" Jul 15 22:12:24.627606 ip-10-0-130-62 kubenswrapper[2578]: I0715 22:12:24.627574 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j7s97_50eee8aa-8304-424c-8525-91b270d118ac/kube-rbac-proxy/0.log"