Sep 18 19:14:12.621974 ip-10-0-129-212 systemd[1]: Starting Kubernetes Kubelet... Sep 18 19:14:16.021708 ip-10-0-129-212 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 19:14:16.021708 ip-10-0-129-212 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 18 19:14:16.021708 ip-10-0-129-212 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 19:14:16.021708 ip-10-0-129-212 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 18 19:14:16.021708 ip-10-0-129-212 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 19:14:16.022589 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.021781 2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 18 19:14:16.027081 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027065 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 19:14:16.027081 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027080 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027085 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027089 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027092 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027094 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027097 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027100 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027102 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027105 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027108 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027117 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027121 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027124 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027126 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027129 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027132 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027135 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027138 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027140 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027143 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 19:14:16.027146 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027146 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027149 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027154 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027158 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027162 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027165 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027168 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027172 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027175 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027179 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027182 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027185 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027188 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027192 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027195 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027198 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027201 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027204 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027207 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 19:14:16.027624 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027210 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027213 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027216 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027220 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027224 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027227 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027230 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027233 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027236 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027239 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027242 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027245 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027248 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027250 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027253 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027256 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027259 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027261 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027264 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 19:14:16.028113 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027267 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027271 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027275 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027277 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027280 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027283 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027286 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027288 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027291 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027294 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027296 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027299 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027302 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027304 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027307 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027310 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027313 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027315 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027318 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 19:14:16.028569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027321 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027324 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027326 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027329 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027332 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027335 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027337 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027340 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027342 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027346 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027349 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027352 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027354 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027357 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027360 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027363 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027365 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027368 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027371 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027373 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.027376 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 19:14:16.029080 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028087 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028094 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028097 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028100 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028103 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028106 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028109 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028112 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028115 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028119 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028122 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028124 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028127 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028130 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028133 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028136 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028138 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028141 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028144 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028147 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 19:14:16.029633 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028150 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028153 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028155 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028158 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028161 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028164 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028167 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028170 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028172 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028175 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028178 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028180 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028183 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028185 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028188 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028191 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028196 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028200 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028204 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 19:14:16.030155 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028207 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028211 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028214 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028218 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028221 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028224 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028227 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028231 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028234 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028236 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028239 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028242 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028245 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028249 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028253 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028256 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028259 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028262 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028265 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028268 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 19:14:16.030631 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028271 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028274 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028279 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028281 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028284 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028287 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028289 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028292 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028296 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028300 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028303 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028305 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028308 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028311 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028313 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028316 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028319 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028321 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028324 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 19:14:16.031139 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028327 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028329 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028332 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028335 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028337 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028340 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028342 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028345 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028348 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028351 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028354 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028356 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028359 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028361 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028365 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028367 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028370 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028372 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028375 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028377 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 19:14:16.031600 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028380 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028762 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028768 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028771 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028774 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028777 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028779 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028782 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028785 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028788 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028791 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028794 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028798 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028801 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028804 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028806 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028809 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028812 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028814 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 19:14:16.032108 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028817 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028820 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028823 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028825 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028828 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028831 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028833 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028836 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028838 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028841 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028843 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028846 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028849 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028851 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028854 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028856 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028859 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028861 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028864 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028866 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 19:14:16.032569 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028870 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028874 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028877 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028882 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028885 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028888 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028891 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028895 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028898 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028901 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028904 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028907 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028910 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028913 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028916 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028919 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028921 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028924 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028926 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 19:14:16.033110 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028929 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028933 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028935 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028938 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028940 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028943 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028946 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028948 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028951 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028953 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028956 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028958 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028961 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028964 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028967 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028970 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028972 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028975 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028978 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028981 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 19:14:16.033576 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028984 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028986 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028989 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028991 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028994 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028996 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.028999 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029001 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029005 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029007 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029010 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029012 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029015 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029017 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029020 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029023 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029025 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029028 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029031 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029033 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 19:14:16.034085 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029036 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029038 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029108 2554 flags.go:64] FLAG: --address="0.0.0.0" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029145 2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029151 2554 flags.go:64] FLAG: --anonymous-auth="true" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029156 2554 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029160 2554 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029166 2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029171 2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029175 2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029180 2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029184 2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029187 2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029191 2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029194 2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029197 2554 flags.go:64] FLAG: --cgroup-root="" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029200 2554 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029203 2554 flags.go:64] FLAG: --client-ca-file="" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029206 2554 flags.go:64] FLAG: --cloud-provider="external" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029209 2554 flags.go:64] FLAG: --cluster-dns="[]" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029214 2554 flags.go:64] FLAG: --cluster-domain="" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029217 2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029220 2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029223 2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 18 19:14:16.034593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029226 2554 flags.go:64] FLAG: --container-log-max-files="5" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029230 2554 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029233 2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029236 2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029240 2554 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029242 2554 flags.go:64] FLAG: --contention-profiling="false" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029245 2554 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029248 2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029252 2554 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029255 2554 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029258 2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029262 2554 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029265 2554 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029267 2554 flags.go:64] FLAG: --enable-load-reader="false" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029271 2554 flags.go:64] FLAG: --enable-server="true" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029276 2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029280 2554 flags.go:64] FLAG: --event-burst="100" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029284 2554 flags.go:64] FLAG: --event-qps="50" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029287 2554 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029290 2554 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029294 2554 flags.go:64] FLAG: --eviction-hard="" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029297 2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029300 2554 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029303 2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029307 2554 flags.go:64] FLAG: --eviction-soft="" Sep 18 19:14:16.035188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029310 2554 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029313 2554 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029316 2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029319 2554 flags.go:64] FLAG: --experimental-mounter-path="" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029322 2554 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029325 2554 flags.go:64] FLAG: --fail-swap-on="true" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029328 2554 flags.go:64] FLAG: --feature-gates="" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029332 2554 flags.go:64] FLAG: --file-check-frequency="20s" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029335 2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029338 2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029341 2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029344 2554 flags.go:64] FLAG: --healthz-port="10248" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029347 2554 flags.go:64] FLAG: --help="false" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029350 2554 flags.go:64] FLAG: --hostname-override="ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029353 2554 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029356 2554 flags.go:64] FLAG: --http-check-frequency="20s" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029361 2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029364 2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029368 2554 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029371 2554 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029374 2554 flags.go:64] FLAG: --image-service-endpoint="" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029377 2554 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029381 2554 flags.go:64] FLAG: --kube-api-burst="100" Sep 18 19:14:16.035806 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029384 2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029387 2554 flags.go:64] FLAG: --kube-api-qps="50" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029390 2554 flags.go:64] FLAG: --kube-reserved="" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029393 2554 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029396 2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029400 2554 flags.go:64] FLAG: --kubelet-cgroups="" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029403 2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029406 2554 flags.go:64] FLAG: --lock-file="" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029409 2554 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029411 2554 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029414 2554 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029420 2554 flags.go:64] FLAG: --log-json-split-stream="false" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029423 2554 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029426 2554 flags.go:64] FLAG: --log-text-split-stream="false" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029429 2554 flags.go:64] FLAG: --logging-format="text" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029431 2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029435 2554 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029438 2554 flags.go:64] FLAG: --manifest-url="" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029440 2554 flags.go:64] FLAG: --manifest-url-header="" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029444 2554 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029447 2554 flags.go:64] FLAG: --max-open-files="1000000" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029451 2554 flags.go:64] FLAG: --max-pods="110" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029455 2554 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029458 2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029460 2554 flags.go:64] FLAG: --memory-manager-policy="None" Sep 18 19:14:16.036404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029465 2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029468 2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029471 2554 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029474 2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029481 2554 flags.go:64] FLAG: --node-status-max-images="50" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029484 2554 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029489 2554 flags.go:64] FLAG: --oom-score-adj="-999" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029493 2554 flags.go:64] FLAG: --pod-cidr="" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029496 2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029501 2554 flags.go:64] FLAG: --pod-manifest-path="" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029504 2554 flags.go:64] FLAG: --pod-max-pids="-1" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029508 2554 flags.go:64] FLAG: --pods-per-core="0" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029511 2554 flags.go:64] FLAG: --port="10250" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029515 2554 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029518 2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c665ebbac896da4b" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029521 2554 flags.go:64] FLAG: --qos-reserved="" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029524 2554 flags.go:64] FLAG: --read-only-port="10255" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029527 2554 flags.go:64] FLAG: --register-node="true" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029530 2554 flags.go:64] FLAG: --register-with-taints="" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029533 2554 flags.go:64] FLAG: --registry-burst="10" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029536 2554 flags.go:64] FLAG: --registry-qps="5" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029539 2554 flags.go:64] FLAG: --reserved-cpus="" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029542 2554 flags.go:64] FLAG: --reserved-memory="" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029546 2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029549 2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 18 19:14:16.037273 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029552 2554 flags.go:64] FLAG: --rotate-certificates="false" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029555 2554 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029558 2554 flags.go:64] FLAG: --runonce="false" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029561 2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029564 2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029567 2554 flags.go:64] FLAG: --seccomp-default="false" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029570 2554 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029574 2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029577 2554 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029581 2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029584 2554 flags.go:64] FLAG: --storage-driver-password="root" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029587 2554 flags.go:64] FLAG: --storage-driver-secure="false" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029590 2554 flags.go:64] FLAG: --storage-driver-table="stats" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029594 2554 flags.go:64] FLAG: --storage-driver-user="root" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029598 2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029601 2554 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029604 2554 flags.go:64] FLAG: --system-cgroups="" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029607 2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029614 2554 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029616 2554 flags.go:64] FLAG: --tls-cert-file="" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029620 2554 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029625 2554 flags.go:64] FLAG: --tls-min-version="" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029628 2554 flags.go:64] FLAG: --tls-private-key-file="" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029631 2554 flags.go:64] FLAG: --topology-manager-policy="none" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029648 2554 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 18 19:14:16.038110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029652 2554 flags.go:64] FLAG: --topology-manager-scope="container" Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029655 2554 flags.go:64] FLAG: --v="2" Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029659 2554 flags.go:64] FLAG: --version="false" Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029663 2554 flags.go:64] FLAG: --vmodule="" Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029667 2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.029671 2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029803 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029807 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029810 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029813 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029816 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029819 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029823 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029825 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029830 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029833 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029836 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029839 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029842 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029844 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029847 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 19:14:16.038823 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029851 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029854 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029857 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029860 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029862 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029865 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029868 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029871 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029874 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029877 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029879 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029882 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029885 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029888 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029891 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029893 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029896 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029899 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029902 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029904 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 19:14:16.039345 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029907 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029910 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029912 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029915 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029918 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029922 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029925 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029927 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029930 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029932 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029935 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029938 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029942 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029944 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029947 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029950 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029953 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029955 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029958 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029961 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 19:14:16.039857 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029964 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029967 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029969 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029972 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029974 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029977 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029980 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029982 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029985 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029988 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029990 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029993 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.029996 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030000 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030003 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030007 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030011 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030015 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030018 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 19:14:16.040349 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030021 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030024 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030027 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030029 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030032 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030037 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030042 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030044 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030047 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030050 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030053 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030056 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030058 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030061 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030065 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030067 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030070 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030073 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030075 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 19:14:16.040839 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030078 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030080 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030083 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030086 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030088 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.030091 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.030550 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.038609 2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.038624 2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.038661 2554 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.038668 2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038725 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038730 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038734 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038738 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 19:14:16.041306 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038741 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038744 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038747 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038750 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038753 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038755 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038758 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038761 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038764 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038768 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038771 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038774 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038777 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038780 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038782 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038785 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038788 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038791 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038794 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 19:14:16.041704 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038802 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038805 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038807 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038810 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038813 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038816 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038820 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038824 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038827 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038831 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038835 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038838 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038841 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038844 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038847 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038849 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038852 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038855 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038858 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038860 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 19:14:16.042194 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038863 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038865 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038868 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038871 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038874 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038877 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038880 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038882 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038885 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038887 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038890 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038893 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038895 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038898 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038901 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038903 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038906 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038909 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038911 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038914 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 19:14:16.042732 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038917 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038919 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038922 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038924 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038927 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038929 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038932 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038934 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038937 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038941 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038944 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038948 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038950 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038953 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038956 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038959 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038963 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038966 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038968 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038971 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 19:14:16.043212 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038973 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038976 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038979 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038981 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038984 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038987 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038989 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038992 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038995 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.038998 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039000 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039003 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039005 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039008 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039011 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 19:14:16.043714 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039014 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.039019 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039147 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039153 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039157 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039160 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039163 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039165 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039169 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039171 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039175 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039177 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039180 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039183 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 19:14:16.044075 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039188 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039191 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039195 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039198 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039201 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039204 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039206 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039209 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039212 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039214 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039217 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039219 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039222 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039225 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039228 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039230 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039233 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039236 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039238 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039241 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 19:14:16.044426 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039243 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039246 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039249 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039252 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039254 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039257 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039260 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039264 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039267 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039271 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039274 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039278 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039281 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039285 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039288 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039291 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039294 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039297 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039300 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 19:14:16.044933 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039302 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039305 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039308 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039310 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039313 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039315 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039318 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039321 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039323 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039328 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039331 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039334 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039337 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039339 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039342 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039345 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039348 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039351 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039354 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039357 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 19:14:16.045391 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039360 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039363 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039366 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039368 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039371 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039374 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039376 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039380 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039382 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039385 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039388 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039391 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039394 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039396 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039399 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039403 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039406 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039409 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039411 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039414 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 19:14:16.045886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039417 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039421 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039423 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039426 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039429 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039431 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039434 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:16.039437 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.039442 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.039554 2554 server.go:962] "Client rotation is on, will bootstrap in background" Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.041619 2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.042593 2554 server.go:1019] "Starting client certificate rotation" Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.042694 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.042725 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 19:14:16.046365 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.044829 2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 19:14:16.046728 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.046065 2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 19:14:16.050683 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.050664 2554 log.go:25] "Validated CRI v1 runtime API" Sep 18 19:14:16.052853 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.052836 2554 log.go:25] "Validated CRI v1 image API" Sep 18 19:14:16.053897 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.053871 2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 18 19:14:16.058360 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.058330 2554 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7fb3c40d-1465-4c70-8481-eeb6730698c0:/dev/nvme0n1p4 e866e730-acbd-479e-98d1-72b34f360520:/dev/nvme0n1p3] Sep 18 19:14:16.058440 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.058358 2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 18 19:14:16.063190 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.063045 2554 manager.go:216] Machine: {Timestamp:2026-09-18 19:14:16.062565748 +0000 UTC m=+1.966913507 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100612 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2974874db75e65747991db0a914929 SystemUUID:ec297487-4db7-5e65-7479-91db0a914929 BootID:f17b09c3-cab0-42d0-b7c6-8c923bea9e4c Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f2:6b:01:e5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f2:6b:01:e5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:b0:d9:b0:ff:db Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 18 19:14:16.063190 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.063179 2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 18 19:14:16.063354 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.063299 2554 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 18 19:14:16.064956 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.064930 2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 18 19:14:16.065137 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.064957 2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-212.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 18 19:14:16.066126 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.066113 2554 topology_manager.go:138] "Creating topology manager with none policy" Sep 18 19:14:16.066183 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.066134 2554 container_manager_linux.go:309] "Creating device plugin manager" Sep 18 19:14:16.066183 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.066146 2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 19:14:16.066183 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.066166 2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 19:14:16.066329 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.066253 2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 18 19:14:16.068714 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.068699 2554 state_mem.go:36] "Initialized new in-memory state store" Sep 18 19:14:16.068847 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.068836 2554 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 18 19:14:16.068903 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.068876 2554 kubelet.go:486] "Attempting to sync node with API server" Sep 18 19:14:16.068903 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.068891 2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 18 19:14:16.068999 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.068912 2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 18 19:14:16.068999 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.068924 2554 kubelet.go:398] "Adding apiserver pod source" Sep 18 19:14:16.068999 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.068959 2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 18 19:14:16.069933 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.069914 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 19:14:16.070026 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.069943 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 19:14:16.072106 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.072082 2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 18 19:14:16.074537 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.074522 2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 18 19:14:16.074604 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.074552 2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 18 19:14:16.081112 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081097 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 18 19:14:16.081180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081116 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 18 19:14:16.081180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081122 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 18 19:14:16.081180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081127 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 18 19:14:16.081180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081136 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 18 19:14:16.081180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081144 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 18 19:14:16.081180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081161 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 18 19:14:16.081180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081166 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 18 19:14:16.081180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081173 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 18 19:14:16.081180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081179 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 18 19:14:16.081414 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081178 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 19:14:16.081414 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081187 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 18 19:14:16.081414 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081204 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 18 19:14:16.081414 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081224 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 18 19:14:16.081414 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.081228 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 18 19:14:16.084621 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.084591 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 18 19:14:16.084714 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.084680 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-212.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 18 19:14:16.085092 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.085080 2554 server.go:1268] "Started kubelet" Sep 18 19:14:16.085198 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.085159 2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 18 19:14:16.085255 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.085183 2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 18 19:14:16.085255 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.085248 2554 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 18 19:14:16.085925 ip-10-0-129-212 systemd[1]: Started Kubernetes Kubelet. Sep 18 19:14:16.086405 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.086395 2554 server.go:310] "Adding debug handlers to kubelet server" Sep 18 19:14:16.086523 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.086395 2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 18 19:14:16.090300 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.090285 2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-212.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 18 19:14:16.091551 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.091537 2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 18 19:14:16.091620 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.091573 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 18 19:14:16.091620 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.091583 2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 18 19:14:16.091759 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.091629 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:16.092178 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.092164 2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 18 19:14:16.092278 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.092168 2554 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 18 19:14:16.092352 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.092338 2554 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 18 19:14:16.092472 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.092458 2554 reconstruct.go:97] "Volume reconstruction finished" Sep 18 19:14:16.092472 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.092472 2554 reconciler.go:29] "Reconciler: start to sync state" Sep 18 19:14:16.092791 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.092773 2554 factory.go:55] Registering systemd factory Sep 18 19:14:16.092857 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.092796 2554 factory.go:223] Registration of the systemd container factory successfully Sep 18 19:14:16.092998 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.092982 2554 factory.go:153] Registering CRI-O factory Sep 18 19:14:16.092998 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.092997 2554 factory.go:223] Registration of the crio container factory successfully Sep 18 19:14:16.093098 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.093069 2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 18 19:14:16.093098 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.093089 2554 factory.go:103] Registering Raw factory Sep 18 19:14:16.093163 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.093119 2554 manager.go:1196] Started watching for new ooms in manager Sep 18 19:14:16.093824 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.093808 2554 manager.go:349] Starting recovery of all containers Sep 18 19:14:16.097145 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.097120 2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-212.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 18 19:14:16.097288 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.097264 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 18 19:14:16.119203 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.119174 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kkkwn" Sep 18 19:14:16.119316 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.119205 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kkkwn" Sep 18 19:14:16.126059 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.125893 2554 manager.go:354] Recovery completed Sep 18 19:14:16.126139 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.124878 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-212.ec2.internal.18d68032728f10f0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-212.ec2.internal,UID:ip-10-0-129-212.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-212.ec2.internal,},FirstTimestamp:2026-09-18 19:14:16.085049584 +0000 UTC m=+1.989397332,LastTimestamp:2026-09-18 19:14:16.085049584 +0000 UTC m=+1.989397332,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-212.ec2.internal,}" Sep 18 19:14:16.130390 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.130378 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 19:14:16.132339 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.132323 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Sep 18 19:14:16.132408 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.132351 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 19:14:16.132408 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.132361 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Sep 18 19:14:16.132993 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.132978 2554 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 18 19:14:16.132993 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.132988 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 19:14:16.132993 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.132993 2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 18 19:14:16.133181 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.133011 2554 state_mem.go:36] "Initialized new in-memory state store" Sep 18 19:14:16.133759 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.133746 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Sep 18 19:14:16.133808 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.133772 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 19:14:16.133808 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.133785 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Sep 18 19:14:16.134023 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.134011 2554 policy_none.go:49] "None policy: Start" Sep 18 19:14:16.134073 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.134030 2554 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 18 19:14:16.134073 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.134043 2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 18 19:14:16.134932 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.134920 2554 policy_none.go:47] "Start" Sep 18 19:14:16.176284 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.176267 2554 manager.go:337] "Starting Device Plugin manager" Sep 18 19:14:16.178558 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.176322 2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 18 19:14:16.178558 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.176335 2554 server.go:87] "Starting device plugin registration server" Sep 18 19:14:16.178558 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.176599 2554 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 18 19:14:16.178558 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.176620 2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 18 19:14:16.178558 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.176803 2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 18 19:14:16.178558 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.176905 2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 18 19:14:16.178558 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.176918 2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 18 19:14:16.178558 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.177752 2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 18 19:14:16.178558 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.177804 2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:16.269576 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.269541 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 18 19:14:16.270764 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.270747 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 18 19:14:16.270875 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.270805 2554 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 18 19:14:16.270875 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.270833 2554 kubelet.go:2448] "Starting kubelet main sync loop" Sep 18 19:14:16.270973 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.270879 2554 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 18 19:14:16.273852 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.273798 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 19:14:16.277405 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.277390 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 19:14:16.278201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.278188 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Sep 18 19:14:16.278268 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.278216 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 19:14:16.278268 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.278226 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Sep 18 19:14:16.278268 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.278246 2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.309999 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.309976 2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.309999 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.310000 2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-212.ec2.internal\": node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:16.333963 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.333940 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:16.371393 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.371369 2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal"] Sep 18 19:14:16.371483 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.371454 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 19:14:16.372450 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.372434 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Sep 18 19:14:16.372536 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.372461 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 19:14:16.372536 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.372470 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Sep 18 19:14:16.373723 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.373711 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 19:14:16.373848 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.373835 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.373848 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.373864 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 19:14:16.374447 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.374432 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Sep 18 19:14:16.374447 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.374439 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Sep 18 19:14:16.374585 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.374457 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 19:14:16.374585 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.374463 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 19:14:16.374585 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.374466 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Sep 18 19:14:16.374585 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.374477 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Sep 18 19:14:16.375423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.375407 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.375569 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.375434 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 19:14:16.376188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.376176 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Sep 18 19:14:16.376251 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.376197 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 19:14:16.376251 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.376214 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Sep 18 19:14:16.393957 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.393935 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/30f01fd6e2b3707a1839770f19ec7ae0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"30f01fd6e2b3707a1839770f19ec7ae0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.394048 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.393962 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30f01fd6e2b3707a1839770f19ec7ae0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"30f01fd6e2b3707a1839770f19ec7ae0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.394048 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.393991 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c6c0e083d554061009b46141d9770d5-config\") pod \"kube-apiserver-proxy-ip-10-0-129-212.ec2.internal\" (UID: \"4c6c0e083d554061009b46141d9770d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.400214 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.400199 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-212.ec2.internal\" not found" node="ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.404515 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.404500 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-212.ec2.internal\" not found" node="ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.434582 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.434554 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:16.494973 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.494947 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/30f01fd6e2b3707a1839770f19ec7ae0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"30f01fd6e2b3707a1839770f19ec7ae0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.494973 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.494974 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30f01fd6e2b3707a1839770f19ec7ae0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"30f01fd6e2b3707a1839770f19ec7ae0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.495112 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.494991 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c6c0e083d554061009b46141d9770d5-config\") pod \"kube-apiserver-proxy-ip-10-0-129-212.ec2.internal\" (UID: \"4c6c0e083d554061009b46141d9770d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.495112 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.495027 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/30f01fd6e2b3707a1839770f19ec7ae0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"30f01fd6e2b3707a1839770f19ec7ae0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.495112 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.495021 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c6c0e083d554061009b46141d9770d5-config\") pod \"kube-apiserver-proxy-ip-10-0-129-212.ec2.internal\" (UID: \"4c6c0e083d554061009b46141d9770d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.495112 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.495049 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30f01fd6e2b3707a1839770f19ec7ae0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"30f01fd6e2b3707a1839770f19ec7ae0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.534929 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.534874 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:16.635882 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.635855 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:16.703793 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.703767 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.708075 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.707957 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Sep 18 19:14:16.736979 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.736955 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:16.837605 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.837523 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:16.938001 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:16.937970 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:16.999150 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:16.999117 2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 19:14:17.038729 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:17.038704 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:17.042866 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.042842 2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 18 19:14:17.042981 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.042964 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 18 19:14:17.043043 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.043001 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 18 19:14:17.125658 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.125563 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 19:09:16 +0000 UTC" deadline="2028-05-04 10:45:26.828826475 +0000 UTC" Sep 18 19:14:17.125658 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.125593 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14247h31m9.703236726s" Sep 18 19:14:17.139732 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:17.139705 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:17.240813 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:17.240787 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:17.306893 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.306861 2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 19:14:17.341003 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:17.340983 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:17.375879 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:17.375806 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30f01fd6e2b3707a1839770f19ec7ae0.slice/crio-28453de3a56f9850c0153d8f3ce985edafb533a97e29e1a6010255acfcbe79ba WatchSource:0}: Error finding container 28453de3a56f9850c0153d8f3ce985edafb533a97e29e1a6010255acfcbe79ba: Status 404 returned error can't find the container with id 28453de3a56f9850c0153d8f3ce985edafb533a97e29e1a6010255acfcbe79ba Sep 18 19:14:17.376210 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:17.376187 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c6c0e083d554061009b46141d9770d5.slice/crio-3fa3bf0a703c3d28fc47714d15096fd7c576b16cdfbbd05d76e89d7d94722bbb WatchSource:0}: Error finding container 3fa3bf0a703c3d28fc47714d15096fd7c576b16cdfbbd05d76e89d7d94722bbb: Status 404 returned error can't find the container with id 3fa3bf0a703c3d28fc47714d15096fd7c576b16cdfbbd05d76e89d7d94722bbb Sep 18 19:14:17.380788 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.380772 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 19:14:17.441593 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:17.441561 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:17.542077 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:17.542033 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Sep 18 19:14:17.545074 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.545056 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 19:14:17.592176 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.592151 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Sep 18 19:14:17.616778 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.616759 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 19:14:17.617776 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.617763 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Sep 18 19:14:17.636321 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:17.636269 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 19:14:18.069804 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.069766 2554 apiserver.go:52] "Watching apiserver" Sep 18 19:14:18.077462 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.077435 2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 18 19:14:18.077870 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.077843 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal","openshift-network-operator/iptables-alerter-89nfk","openshift-ovn-kubernetes/ovnkube-node-bqqtf","kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj","openshift-dns/node-resolver-v99gh","openshift-image-registry/node-ca-9fpjr","openshift-multus/multus-additional-cni-plugins-qrv5j","openshift-multus/multus-n5qn8","openshift-multus/network-metrics-daemon-dvkb2","openshift-network-diagnostics/network-check-target-2jhvf","kube-system/global-pull-secret-syncer-nphvd","kube-system/konnectivity-agent-7kfxm","openshift-cluster-node-tuning-operator/tuned-jrdfc"] Sep 18 19:14:18.080206 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.079900 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:18.080206 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.079978 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:18.081777 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.081756 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:18.083766 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.083745 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.085748 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.085726 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.086905 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.086887 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 18 19:14:18.087167 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.087152 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 18 19:14:18.087919 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.087671 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.089867 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.089847 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v99gh" Sep 18 19:14:18.092063 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.092045 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.092508 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.092488 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 18 19:14:18.094410 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.094393 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9fpjr" Sep 18 19:14:18.096520 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.096501 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-89nfk" Sep 18 19:14:18.099751 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.099729 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.102010 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.101937 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:18.102010 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.102000 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:18.102124 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102016 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-sysctl-conf\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.102124 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102041 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-host\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.102124 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102063 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-cnibin\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.102124 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102084 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-run-netns\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.102124 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102106 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-run-ovn\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.102286 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102126 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51fae194-f7e3-415c-ad89-4a025211c4b8-tmp\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.102286 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102147 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-system-cni-dir\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.102286 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102177 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-slash\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.102286 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102241 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-cni-bin\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.102286 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102271 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-ovnkube-config\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.102518 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102304 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc2c0c9b-3578-46cf-a91e-b3eeef557f64-host\") pod \"node-ca-9fpjr\" (UID: \"bc2c0c9b-3578-46cf-a91e-b3eeef557f64\") " pod="openshift-image-registry/node-ca-9fpjr" Sep 18 19:14:18.102518 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102347 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54jc6\" (UniqueName: \"kubernetes.io/projected/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-kube-api-access-54jc6\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.102518 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102376 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d0e6fa6e-811a-4d85-a190-a27c5563dfd9-konnectivity-ca\") pod \"konnectivity-agent-7kfxm\" (UID: \"d0e6fa6e-811a-4d85-a190-a27c5563dfd9\") " pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:18.102518 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102398 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-multus-socket-dir-parent\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.102518 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102458 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d0ae961b-b362-4061-8067-faf028e26649-multus-daemon-config\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.102518 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102490 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhm79\" (UniqueName: \"kubernetes.io/projected/0899c67e-8d96-4fa0-b310-b6641c91127b-kube-api-access-vhm79\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:18.102518 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102515 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8plsf\" (UniqueName: \"kubernetes.io/projected/bc2c0c9b-3578-46cf-a91e-b3eeef557f64-kube-api-access-8plsf\") pod \"node-ca-9fpjr\" (UID: \"bc2c0c9b-3578-46cf-a91e-b3eeef557f64\") " pod="openshift-image-registry/node-ca-9fpjr" Sep 18 19:14:18.102829 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102560 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-run\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.102829 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102590 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-lib-modules\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.102829 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102616 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f6f93841-9b28-41ca-aab2-4caefd768842-hosts-file\") pod \"node-resolver-v99gh\" (UID: \"f6f93841-9b28-41ca-aab2-4caefd768842\") " pod="openshift-dns/node-resolver-v99gh" Sep 18 19:14:18.102829 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102653 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-kubelet\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.102829 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102678 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-etc-openvswitch\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.102829 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102700 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.102829 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102725 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-var-lib-cni-multus\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.102829 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102773 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-multus-cni-dir\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.102829 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102800 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-node-log\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.102829 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102824 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-log-socket\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102848 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102873 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-kubernetes\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102924 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-tuned\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102947 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-hostroot\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102964 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.102978 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-etc-selinux\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103011 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-run-multus-certs\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103046 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bc2c0c9b-3578-46cf-a91e-b3eeef557f64-serviceca\") pod \"node-ca-9fpjr\" (UID: \"bc2c0c9b-3578-46cf-a91e-b3eeef557f64\") " pod="openshift-image-registry/node-ca-9fpjr" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103088 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0ae961b-b362-4061-8067-faf028e26649-cni-binary-copy\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103116 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-sysconfig\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103148 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-systemd-units\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.103172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103175 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-sysctl-d\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103198 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7hjp\" (UniqueName: \"kubernetes.io/projected/51fae194-f7e3-415c-ad89-4a025211c4b8-kube-api-access-r7hjp\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103228 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-registration-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103252 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-run-k8s-cni-cncf-io\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103274 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-ovnkube-script-lib\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103297 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-run-systemd\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103330 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-run-ovn-kubernetes\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103352 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-cni-netd\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103375 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-ovn-node-metrics-cert\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103397 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-multus-conf-dir\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103418 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-env-overrides\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103441 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-modprobe-d\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103461 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-socket-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.103736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103495 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-run-openvswitch\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103799 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-sys\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103830 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f93841-9b28-41ca-aab2-4caefd768842-tmp-dir\") pod \"node-resolver-v99gh\" (UID: \"f6f93841-9b28-41ca-aab2-4caefd768842\") " pod="openshift-dns/node-resolver-v99gh" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103861 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lkv2\" (UniqueName: \"kubernetes.io/projected/f6f93841-9b28-41ca-aab2-4caefd768842-kube-api-access-4lkv2\") pod \"node-resolver-v99gh\" (UID: \"f6f93841-9b28-41ca-aab2-4caefd768842\") " pod="openshift-dns/node-resolver-v99gh" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103906 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-var-lib-cni-bin\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103929 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d0e6fa6e-811a-4d85-a190-a27c5563dfd9-agent-certs\") pod \"konnectivity-agent-7kfxm\" (UID: \"d0e6fa6e-811a-4d85-a190-a27c5563dfd9\") " pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103963 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-sys-fs\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.103989 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr9bz\" (UniqueName: \"kubernetes.io/projected/484badbc-902c-4577-a5cd-8893ea859178-kube-api-access-fr9bz\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.104011 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-var-lib-kubelet\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.104035 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-etc-kubernetes\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.104060 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-var-lib-openvswitch\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.104083 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-systemd\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.104112 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-var-lib-kubelet\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.104126 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.104141 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-device-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.104168 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-run-netns\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.104183 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:18.104340 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.104195 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g67sb\" (UniqueName: \"kubernetes.io/projected/d0ae961b-b362-4061-8067-faf028e26649-kube-api-access-g67sb\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.108682 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.107199 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 18 19:14:18.108682 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.107269 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 18 19:14:18.108682 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.107453 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 18 19:14:18.108682 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.107536 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 18 19:14:18.108682 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.107689 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kgzbw\"" Sep 18 19:14:18.108682 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.107761 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 18 19:14:18.108682 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.107887 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 18 19:14:18.108682 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.107997 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 18 19:14:18.108682 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.108118 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 18 19:14:18.108682 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.108159 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 18 19:14:18.108682 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.108369 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.109567 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.109617 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.109841 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.109891 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9f6lc\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.110073 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tnnmq\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.110090 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.110092 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.110288 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.110355 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.109843 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kr9kd\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.110581 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.110782 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xzmsw\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.110948 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.111138 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ch9ws\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.111312 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.111476 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n5qr9\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.111613 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.111796 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.111989 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rrlpz\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.112164 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.112366 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fr685\"" Sep 18 19:14:18.113201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.112516 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 18 19:14:18.120922 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.120903 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 18 19:14:18.136283 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.136125 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 19:14:18.152307 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.152284 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x5lkj" Sep 18 19:14:18.168428 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.168389 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x5lkj" Sep 18 19:14:18.193379 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.193350 2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 18 19:14:18.205041 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.204976 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-registration-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.205041 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205024 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-run-k8s-cni-cncf-io\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.205191 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205051 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-ovnkube-script-lib\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205191 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205067 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-registration-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.205191 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205111 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-run-k8s-cni-cncf-io\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.205191 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205156 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-run-systemd\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205191 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205186 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-run-ovn-kubernetes\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205212 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-cni-netd\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205235 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-ovn-node-metrics-cert\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205258 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-multus-conf-dir\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.205423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205263 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-run-systemd\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205282 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-run-ovn-kubernetes\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205316 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-env-overrides\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205330 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-multus-conf-dir\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.205423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205358 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-modprobe-d\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.205423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205366 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-cni-netd\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205383 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-socket-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.205423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205414 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205442 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205467 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d7623d04-8395-4bc7-b628-d82e5e22b847-dbus\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205491 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-run-openvswitch\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205508 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-modprobe-d\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205513 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-sys\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205546 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f93841-9b28-41ca-aab2-4caefd768842-tmp-dir\") pod \"node-resolver-v99gh\" (UID: \"f6f93841-9b28-41ca-aab2-4caefd768842\") " pod="openshift-dns/node-resolver-v99gh" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205560 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-sys\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205570 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lkv2\" (UniqueName: \"kubernetes.io/projected/f6f93841-9b28-41ca-aab2-4caefd768842-kube-api-access-4lkv2\") pod \"node-resolver-v99gh\" (UID: \"f6f93841-9b28-41ca-aab2-4caefd768842\") " pod="openshift-dns/node-resolver-v99gh" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205601 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-var-lib-cni-bin\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205627 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-system-cni-dir\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205610 2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205667 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-ovnkube-script-lib\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205675 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swxhx\" (UniqueName: \"kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx\") pod \"network-check-target-2jhvf\" (UID: \"a0a9191c-4b4d-4bfc-8e5d-a2675b620375\") " pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205728 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d0e6fa6e-811a-4d85-a190-a27c5563dfd9-agent-certs\") pod \"konnectivity-agent-7kfxm\" (UID: \"d0e6fa6e-811a-4d85-a190-a27c5563dfd9\") " pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205743 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-sys-fs\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205751 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-env-overrides\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.205969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205759 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr9bz\" (UniqueName: \"kubernetes.io/projected/484badbc-902c-4577-a5cd-8893ea859178-kube-api-access-fr9bz\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205775 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-var-lib-kubelet\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205772 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-var-lib-cni-bin\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205791 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-etc-kubernetes\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205819 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-etc-kubernetes\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205826 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d7623d04-8395-4bc7-b628-d82e5e22b847-kubelet-config\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205741 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-run-openvswitch\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205854 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-var-lib-openvswitch\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205883 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-var-lib-openvswitch\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205906 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-socket-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205918 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-systemd\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.205947 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-var-lib-kubelet\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206007 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f93841-9b28-41ca-aab2-4caefd768842-tmp-dir\") pod \"node-resolver-v99gh\" (UID: \"f6f93841-9b28-41ca-aab2-4caefd768842\") " pod="openshift-dns/node-resolver-v99gh" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206064 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-systemd\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206061 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-var-lib-kubelet\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206086 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-device-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206106 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-sys-fs\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206122 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-run-netns\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.206947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206159 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-device-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206161 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-var-lib-kubelet\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206192 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g67sb\" (UniqueName: \"kubernetes.io/projected/d0ae961b-b362-4061-8067-faf028e26649-kube-api-access-g67sb\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206212 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-run-netns\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206250 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-sysctl-conf\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206274 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-host\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206297 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-cnibin\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206349 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-host\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206357 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ed44b0a-93ec-4288-b50b-d0fa7427410c-host-slash\") pod \"iptables-alerter-89nfk\" (UID: \"8ed44b0a-93ec-4288-b50b-d0fa7427410c\") " pod="openshift-network-operator/iptables-alerter-89nfk" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206391 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-cnibin\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206393 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-sysctl-conf\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206393 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-run-netns\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206429 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-run-netns\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206439 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-run-ovn\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206459 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51fae194-f7e3-415c-ad89-4a025211c4b8-tmp\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206470 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-run-ovn\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206479 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-system-cni-dir\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206498 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-slash\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.207801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206520 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-cni-bin\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206543 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-ovnkube-config\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206547 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-system-cni-dir\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206565 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc2c0c9b-3578-46cf-a91e-b3eeef557f64-host\") pod \"node-ca-9fpjr\" (UID: \"bc2c0c9b-3578-46cf-a91e-b3eeef557f64\") " pod="openshift-image-registry/node-ca-9fpjr" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206586 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-cni-bin\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206590 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqlzp\" (UniqueName: \"kubernetes.io/projected/8ed44b0a-93ec-4288-b50b-d0fa7427410c-kube-api-access-rqlzp\") pod \"iptables-alerter-89nfk\" (UID: \"8ed44b0a-93ec-4288-b50b-d0fa7427410c\") " pod="openshift-network-operator/iptables-alerter-89nfk" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206615 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-slash\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206617 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-cni-binary-copy\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206656 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54jc6\" (UniqueName: \"kubernetes.io/projected/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-kube-api-access-54jc6\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206663 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc2c0c9b-3578-46cf-a91e-b3eeef557f64-host\") pod \"node-ca-9fpjr\" (UID: \"bc2c0c9b-3578-46cf-a91e-b3eeef557f64\") " pod="openshift-image-registry/node-ca-9fpjr" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206683 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d0e6fa6e-811a-4d85-a190-a27c5563dfd9-konnectivity-ca\") pod \"konnectivity-agent-7kfxm\" (UID: \"d0e6fa6e-811a-4d85-a190-a27c5563dfd9\") " pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206708 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-cnibin\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206730 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-multus-socket-dir-parent\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206751 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d0ae961b-b362-4061-8067-faf028e26649-multus-daemon-config\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206771 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhm79\" (UniqueName: \"kubernetes.io/projected/0899c67e-8d96-4fa0-b310-b6641c91127b-kube-api-access-vhm79\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206797 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8plsf\" (UniqueName: \"kubernetes.io/projected/bc2c0c9b-3578-46cf-a91e-b3eeef557f64-kube-api-access-8plsf\") pod \"node-ca-9fpjr\" (UID: \"bc2c0c9b-3578-46cf-a91e-b3eeef557f64\") " pod="openshift-image-registry/node-ca-9fpjr" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206817 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-run\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.208666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206862 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-lib-modules\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206897 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f6f93841-9b28-41ca-aab2-4caefd768842-hosts-file\") pod \"node-resolver-v99gh\" (UID: \"f6f93841-9b28-41ca-aab2-4caefd768842\") " pod="openshift-dns/node-resolver-v99gh" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206926 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ed44b0a-93ec-4288-b50b-d0fa7427410c-iptables-alerter-script\") pod \"iptables-alerter-89nfk\" (UID: \"8ed44b0a-93ec-4288-b50b-d0fa7427410c\") " pod="openshift-network-operator/iptables-alerter-89nfk" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206954 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206977 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k99xl\" (UniqueName: \"kubernetes.io/projected/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-kube-api-access-k99xl\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207006 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-kubelet\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207035 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-etc-openvswitch\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207061 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207084 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-var-lib-cni-multus\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207108 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207184 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-kubelet\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207184 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-multus-socket-dir-parent\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207197 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-ovnkube-config\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.206870 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-run\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207230 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d0e6fa6e-811a-4d85-a190-a27c5563dfd9-konnectivity-ca\") pod \"konnectivity-agent-7kfxm\" (UID: \"d0e6fa6e-811a-4d85-a190-a27c5563dfd9\") " pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207227 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-etc-openvswitch\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207269 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.209539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207276 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f6f93841-9b28-41ca-aab2-4caefd768842-hosts-file\") pod \"node-resolver-v99gh\" (UID: \"f6f93841-9b28-41ca-aab2-4caefd768842\") " pod="openshift-dns/node-resolver-v99gh" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207297 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-multus-cni-dir\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207312 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-var-lib-cni-multus\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207325 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-node-log\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207347 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-log-socket\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207371 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207390 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-node-log\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207394 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d0ae961b-b362-4061-8067-faf028e26649-multus-daemon-config\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207409 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-multus-cni-dir\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207397 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-kubernetes\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207434 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-log-socket\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207440 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-kubernetes\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207449 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-tuned\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207465 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-lib-modules\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207451 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207475 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-hostroot\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207519 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207518 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-hostroot\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.210293 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.207586 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207609 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-etc-selinux\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.207670 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs podName:0899c67e-8d96-4fa0-b310-b6641c91127b nodeName:}" failed. No retries permitted until 2026-09-18 19:14:18.707625218 +0000 UTC m=+4.611972975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs") pod "network-metrics-daemon-dvkb2" (UID: "0899c67e-8d96-4fa0-b310-b6641c91127b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207688 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-run-multus-certs\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207714 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/484badbc-902c-4577-a5cd-8893ea859178-etc-selinux\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207722 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bc2c0c9b-3578-46cf-a91e-b3eeef557f64-serviceca\") pod \"node-ca-9fpjr\" (UID: \"bc2c0c9b-3578-46cf-a91e-b3eeef557f64\") " pod="openshift-image-registry/node-ca-9fpjr" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207756 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d0ae961b-b362-4061-8067-faf028e26649-host-run-multus-certs\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207789 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0ae961b-b362-4061-8067-faf028e26649-cni-binary-copy\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207813 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-sysconfig\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207841 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-systemd-units\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207865 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-sysctl-d\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207888 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7hjp\" (UniqueName: \"kubernetes.io/projected/51fae194-f7e3-415c-ad89-4a025211c4b8-kube-api-access-r7hjp\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.207893 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-sysconfig\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.208019 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-systemd-units\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.208112 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-sysctl-d\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.208113 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bc2c0c9b-3578-46cf-a91e-b3eeef557f64-serviceca\") pod \"node-ca-9fpjr\" (UID: \"bc2c0c9b-3578-46cf-a91e-b3eeef557f64\") " pod="openshift-image-registry/node-ca-9fpjr" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.208300 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0ae961b-b362-4061-8067-faf028e26649-cni-binary-copy\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.211097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.209051 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-ovn-node-metrics-cert\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.211848 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.209218 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d0e6fa6e-811a-4d85-a190-a27c5563dfd9-agent-certs\") pod \"konnectivity-agent-7kfxm\" (UID: \"d0e6fa6e-811a-4d85-a190-a27c5563dfd9\") " pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:18.211848 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.210056 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51fae194-f7e3-415c-ad89-4a025211c4b8-tmp\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.211848 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.210561 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/51fae194-f7e3-415c-ad89-4a025211c4b8-etc-tuned\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.233653 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.233614 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lkv2\" (UniqueName: \"kubernetes.io/projected/f6f93841-9b28-41ca-aab2-4caefd768842-kube-api-access-4lkv2\") pod \"node-resolver-v99gh\" (UID: \"f6f93841-9b28-41ca-aab2-4caefd768842\") " pod="openshift-dns/node-resolver-v99gh" Sep 18 19:14:18.245753 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.245687 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhm79\" (UniqueName: \"kubernetes.io/projected/0899c67e-8d96-4fa0-b310-b6641c91127b-kube-api-access-vhm79\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:18.245753 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.245696 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7hjp\" (UniqueName: \"kubernetes.io/projected/51fae194-f7e3-415c-ad89-4a025211c4b8-kube-api-access-r7hjp\") pod \"tuned-jrdfc\" (UID: \"51fae194-f7e3-415c-ad89-4a025211c4b8\") " pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.250727 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.250702 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54jc6\" (UniqueName: \"kubernetes.io/projected/6e2f8a53-2ebd-4d26-9731-0d77648dfcf4-kube-api-access-54jc6\") pod \"ovnkube-node-bqqtf\" (UID: \"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.251727 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.251702 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8plsf\" (UniqueName: \"kubernetes.io/projected/bc2c0c9b-3578-46cf-a91e-b3eeef557f64-kube-api-access-8plsf\") pod \"node-ca-9fpjr\" (UID: \"bc2c0c9b-3578-46cf-a91e-b3eeef557f64\") " pod="openshift-image-registry/node-ca-9fpjr" Sep 18 19:14:18.252849 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.252826 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g67sb\" (UniqueName: \"kubernetes.io/projected/d0ae961b-b362-4061-8067-faf028e26649-kube-api-access-g67sb\") pod \"multus-n5qn8\" (UID: \"d0ae961b-b362-4061-8067-faf028e26649\") " pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.259142 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.259121 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr9bz\" (UniqueName: \"kubernetes.io/projected/484badbc-902c-4577-a5cd-8893ea859178-kube-api-access-fr9bz\") pod \"aws-ebs-csi-driver-node-g5ktj\" (UID: \"484badbc-902c-4577-a5cd-8893ea859178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.275984 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.275909 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" event={"ID":"4c6c0e083d554061009b46141d9770d5","Type":"ContainerStarted","Data":"3fa3bf0a703c3d28fc47714d15096fd7c576b16cdfbbd05d76e89d7d94722bbb"} Sep 18 19:14:18.277124 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.277099 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" event={"ID":"30f01fd6e2b3707a1839770f19ec7ae0","Type":"ContainerStarted","Data":"28453de3a56f9850c0153d8f3ce985edafb533a97e29e1a6010255acfcbe79ba"} Sep 18 19:14:18.308494 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308463 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:18.308673 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308532 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.308673 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308567 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.308673 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308595 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d7623d04-8395-4bc7-b628-d82e5e22b847-dbus\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:18.308673 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308627 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-system-cni-dir\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.308673 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308658 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swxhx\" (UniqueName: \"kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx\") pod \"network-check-target-2jhvf\" (UID: \"a0a9191c-4b4d-4bfc-8e5d-a2675b620375\") " pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:18.308983 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308687 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d7623d04-8395-4bc7-b628-d82e5e22b847-kubelet-config\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:18.308983 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308707 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ed44b0a-93ec-4288-b50b-d0fa7427410c-host-slash\") pod \"iptables-alerter-89nfk\" (UID: \"8ed44b0a-93ec-4288-b50b-d0fa7427410c\") " pod="openshift-network-operator/iptables-alerter-89nfk" Sep 18 19:14:18.308983 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308735 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqlzp\" (UniqueName: \"kubernetes.io/projected/8ed44b0a-93ec-4288-b50b-d0fa7427410c-kube-api-access-rqlzp\") pod \"iptables-alerter-89nfk\" (UID: \"8ed44b0a-93ec-4288-b50b-d0fa7427410c\") " pod="openshift-network-operator/iptables-alerter-89nfk" Sep 18 19:14:18.308983 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308763 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-cni-binary-copy\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.308983 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308778 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-cnibin\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.308983 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308810 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ed44b0a-93ec-4288-b50b-d0fa7427410c-iptables-alerter-script\") pod \"iptables-alerter-89nfk\" (UID: \"8ed44b0a-93ec-4288-b50b-d0fa7427410c\") " pod="openshift-network-operator/iptables-alerter-89nfk" Sep 18 19:14:18.308983 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308836 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.308983 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.308850 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k99xl\" (UniqueName: \"kubernetes.io/projected/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-kube-api-access-k99xl\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.309386 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.309133 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:18.309386 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.309183 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret podName:d7623d04-8395-4bc7-b628-d82e5e22b847 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:18.809170127 +0000 UTC m=+4.713517862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret") pod "global-pull-secret-syncer-nphvd" (UID: "d7623d04-8395-4bc7-b628-d82e5e22b847") : object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:18.309818 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.309792 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ed44b0a-93ec-4288-b50b-d0fa7427410c-host-slash\") pod \"iptables-alerter-89nfk\" (UID: \"8ed44b0a-93ec-4288-b50b-d0fa7427410c\") " pod="openshift-network-operator/iptables-alerter-89nfk" Sep 18 19:14:18.309901 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.309887 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d7623d04-8395-4bc7-b628-d82e5e22b847-kubelet-config\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:18.309901 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.309796 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-system-cni-dir\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.310006 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.309796 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.310006 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.309957 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d7623d04-8395-4bc7-b628-d82e5e22b847-dbus\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:18.310109 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.310015 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-cnibin\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.310190 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.310172 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.310325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.310256 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.310422 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.310398 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ed44b0a-93ec-4288-b50b-d0fa7427410c-iptables-alerter-script\") pod \"iptables-alerter-89nfk\" (UID: \"8ed44b0a-93ec-4288-b50b-d0fa7427410c\") " pod="openshift-network-operator/iptables-alerter-89nfk" Sep 18 19:14:18.310515 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.310424 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-cni-binary-copy\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.319081 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.319057 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 19:14:18.319197 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.319087 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 19:14:18.319197 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.319106 2554 projected.go:196] Error preparing data for projected volume kube-api-access-swxhx for pod openshift-network-diagnostics/network-check-target-2jhvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:18.319197 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.319163 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx podName:a0a9191c-4b4d-4bfc-8e5d-a2675b620375 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:18.819149677 +0000 UTC m=+4.723497413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-swxhx" (UniqueName: "kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx") pod "network-check-target-2jhvf" (UID: "a0a9191c-4b4d-4bfc-8e5d-a2675b620375") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:18.320538 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.320514 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k99xl\" (UniqueName: \"kubernetes.io/projected/059c4ca0-dcb8-4d39-b3b5-0dfad7944b18-kube-api-access-k99xl\") pod \"multus-additional-cni-plugins-qrv5j\" (UID: \"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18\") " pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.322202 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.322174 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqlzp\" (UniqueName: \"kubernetes.io/projected/8ed44b0a-93ec-4288-b50b-d0fa7427410c-kube-api-access-rqlzp\") pod \"iptables-alerter-89nfk\" (UID: \"8ed44b0a-93ec-4288-b50b-d0fa7427410c\") " pod="openshift-network-operator/iptables-alerter-89nfk" Sep 18 19:14:18.393007 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.392975 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:18.401724 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.401701 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:18.412309 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.412288 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" Sep 18 19:14:18.418681 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.418662 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" Sep 18 19:14:18.426325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.426305 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v99gh" Sep 18 19:14:18.435879 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.435855 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n5qn8" Sep 18 19:14:18.442271 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.442248 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9fpjr" Sep 18 19:14:18.450745 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.450725 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-89nfk" Sep 18 19:14:18.457188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.457131 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" Sep 18 19:14:18.558686 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.558660 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 19:14:18.711438 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.711356 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:18.711583 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.711530 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:18.711631 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.711613 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs podName:0899c67e-8d96-4fa0-b310-b6641c91127b nodeName:}" failed. No retries permitted until 2026-09-18 19:14:19.711592166 +0000 UTC m=+5.615939912 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs") pod "network-metrics-daemon-dvkb2" (UID: "0899c67e-8d96-4fa0-b310-b6641c91127b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:18.812426 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.812377 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:18.812583 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.812534 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:18.812659 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.812614 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret podName:d7623d04-8395-4bc7-b628-d82e5e22b847 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:19.812592959 +0000 UTC m=+5.716940711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret") pod "global-pull-secret-syncer-nphvd" (UID: "d7623d04-8395-4bc7-b628-d82e5e22b847") : object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:18.913248 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:18.913208 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swxhx\" (UniqueName: \"kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx\") pod \"network-check-target-2jhvf\" (UID: \"a0a9191c-4b4d-4bfc-8e5d-a2675b620375\") " pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:18.913400 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.913353 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 19:14:18.913400 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.913369 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 19:14:18.913400 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.913379 2554 projected.go:196] Error preparing data for projected volume kube-api-access-swxhx for pod openshift-network-diagnostics/network-check-target-2jhvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:18.913508 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:18.913430 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx podName:a0a9191c-4b4d-4bfc-8e5d-a2675b620375 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:19.913415847 +0000 UTC m=+5.817763594 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-swxhx" (UniqueName: "kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx") pod "network-check-target-2jhvf" (UID: "a0a9191c-4b4d-4bfc-8e5d-a2675b620375") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:19.171622 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:19.171524 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 19:09:18 +0000 UTC" deadline="2028-05-16 15:18:03.048401268 +0000 UTC" Sep 18 19:14:19.171622 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:19.171570 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14540h3m43.876834975s" Sep 18 19:14:19.300753 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:19.300710 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed44b0a_93ec_4288_b50b_d0fa7427410c.slice/crio-3cf694a94a61edff952a445af2d456bd7843fbad26a89f92df27f0e412b89f1d WatchSource:0}: Error finding container 3cf694a94a61edff952a445af2d456bd7843fbad26a89f92df27f0e412b89f1d: Status 404 returned error can't find the container with id 3cf694a94a61edff952a445af2d456bd7843fbad26a89f92df27f0e412b89f1d Sep 18 19:14:19.302190 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:19.302166 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc2c0c9b_3578_46cf_a91e_b3eeef557f64.slice/crio-42ba7f3116527a721e7d8ea2edaa115d764d3804a969362ef56aa84934a35a82 WatchSource:0}: Error finding container 42ba7f3116527a721e7d8ea2edaa115d764d3804a969362ef56aa84934a35a82: Status 404 returned error can't find the container with id 42ba7f3116527a721e7d8ea2edaa115d764d3804a969362ef56aa84934a35a82 Sep 18 19:14:19.303752 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:19.303700 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484badbc_902c_4577_a5cd_8893ea859178.slice/crio-e096df6d2578e9c66b8f6cfb11430240426ad514ff250cc1914d506d1c7e4395 WatchSource:0}: Error finding container e096df6d2578e9c66b8f6cfb11430240426ad514ff250cc1914d506d1c7e4395: Status 404 returned error can't find the container with id e096df6d2578e9c66b8f6cfb11430240426ad514ff250cc1914d506d1c7e4395 Sep 18 19:14:19.306671 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:19.305985 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059c4ca0_dcb8_4d39_b3b5_0dfad7944b18.slice/crio-6a126a6b48a90420bde3a57e00e2f7852ca6481d2f96e1d2eef5cab9c96dcef6 WatchSource:0}: Error finding container 6a126a6b48a90420bde3a57e00e2f7852ca6481d2f96e1d2eef5cab9c96dcef6: Status 404 returned error can't find the container with id 6a126a6b48a90420bde3a57e00e2f7852ca6481d2f96e1d2eef5cab9c96dcef6 Sep 18 19:14:19.306671 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:19.306210 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51fae194_f7e3_415c_ad89_4a025211c4b8.slice/crio-9f82bf4bafbce8d9455f0e642b97acb6e7c6e259adf794f56f48f3ab246f699f WatchSource:0}: Error finding container 9f82bf4bafbce8d9455f0e642b97acb6e7c6e259adf794f56f48f3ab246f699f: Status 404 returned error can't find the container with id 9f82bf4bafbce8d9455f0e642b97acb6e7c6e259adf794f56f48f3ab246f699f Sep 18 19:14:19.307144 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:19.306979 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e6fa6e_811a_4d85_a190_a27c5563dfd9.slice/crio-b217033f32fe3dacbab376c0338b70af9b3473fee9798c2fb0549160574693e2 WatchSource:0}: Error finding container b217033f32fe3dacbab376c0338b70af9b3473fee9798c2fb0549160574693e2: Status 404 returned error can't find the container with id b217033f32fe3dacbab376c0338b70af9b3473fee9798c2fb0549160574693e2 Sep 18 19:14:19.308065 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:19.307957 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e2f8a53_2ebd_4d26_9731_0d77648dfcf4.slice/crio-eb9e1d17762893cd7744cf40591741a6af02a3655be27f6f784dc60ea9562aca WatchSource:0}: Error finding container eb9e1d17762893cd7744cf40591741a6af02a3655be27f6f784dc60ea9562aca: Status 404 returned error can't find the container with id eb9e1d17762893cd7744cf40591741a6af02a3655be27f6f784dc60ea9562aca Sep 18 19:14:19.309240 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:19.309115 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ae961b_b362_4061_8067_faf028e26649.slice/crio-dda2bc02ede0703003cd618845cf23458cbbcd1096d8529cf384f0f520f0ce8e WatchSource:0}: Error finding container dda2bc02ede0703003cd618845cf23458cbbcd1096d8529cf384f0f520f0ce8e: Status 404 returned error can't find the container with id dda2bc02ede0703003cd618845cf23458cbbcd1096d8529cf384f0f520f0ce8e Sep 18 19:14:19.309834 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:19.309743 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f93841_9b28_41ca_aab2_4caefd768842.slice/crio-d8869be32695cd436278a8f637bf5e38f2470deface652e6018eecd9e0b5f458 WatchSource:0}: Error finding container d8869be32695cd436278a8f637bf5e38f2470deface652e6018eecd9e0b5f458: Status 404 returned error can't find the container with id d8869be32695cd436278a8f637bf5e38f2470deface652e6018eecd9e0b5f458 Sep 18 19:14:19.719153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:19.719112 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:19.719319 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:19.719282 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:19.719384 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:19.719346 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs podName:0899c67e-8d96-4fa0-b310-b6641c91127b nodeName:}" failed. No retries permitted until 2026-09-18 19:14:21.719326034 +0000 UTC m=+7.623673788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs") pod "network-metrics-daemon-dvkb2" (UID: "0899c67e-8d96-4fa0-b310-b6641c91127b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:19.819700 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:19.819608 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:19.819862 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:19.819784 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:19.819862 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:19.819837 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret podName:d7623d04-8395-4bc7-b628-d82e5e22b847 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:21.819820598 +0000 UTC m=+7.724168346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret") pod "global-pull-secret-syncer-nphvd" (UID: "d7623d04-8395-4bc7-b628-d82e5e22b847") : object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:19.920574 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:19.920541 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swxhx\" (UniqueName: \"kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx\") pod \"network-check-target-2jhvf\" (UID: \"a0a9191c-4b4d-4bfc-8e5d-a2675b620375\") " pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:19.920762 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:19.920699 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 19:14:19.920762 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:19.920717 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 19:14:19.920762 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:19.920730 2554 projected.go:196] Error preparing data for projected volume kube-api-access-swxhx for pod openshift-network-diagnostics/network-check-target-2jhvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:19.920961 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:19.920792 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx podName:a0a9191c-4b4d-4bfc-8e5d-a2675b620375 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:21.920774707 +0000 UTC m=+7.825122452 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-swxhx" (UniqueName: "kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx") pod "network-check-target-2jhvf" (UID: "a0a9191c-4b4d-4bfc-8e5d-a2675b620375") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:19.956549 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:19.956506 2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 19:14:20.172541 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.172452 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 19:09:18 +0000 UTC" deadline="2028-05-12 23:33:02.854069245 +0000 UTC" Sep 18 19:14:20.172541 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.172492 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14452h18m42.6815823s" Sep 18 19:14:20.281860 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.281835 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:20.282029 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:20.281967 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:20.282391 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.282367 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:20.282500 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:20.282480 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:20.282625 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.282549 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:20.282625 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:20.282617 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:20.290947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.290920 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" event={"ID":"4c6c0e083d554061009b46141d9770d5","Type":"ContainerStarted","Data":"b4f16dc86308dd96c6638edcd0cea087656d437a4d93ebae8fc4a7da7e616543"} Sep 18 19:14:20.296799 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.296755 2554 generic.go:356] "Generic (PLEG): container finished" podID="30f01fd6e2b3707a1839770f19ec7ae0" containerID="a061d89b8cc8cb6048add4480baf66d593453e55ccf00ce6ebde44cbec98ced8" exitCode=0 Sep 18 19:14:20.297180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.297149 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" event={"ID":"30f01fd6e2b3707a1839770f19ec7ae0","Type":"ContainerDied","Data":"a061d89b8cc8cb6048add4480baf66d593453e55ccf00ce6ebde44cbec98ced8"} Sep 18 19:14:20.307556 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.305367 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v99gh" event={"ID":"f6f93841-9b28-41ca-aab2-4caefd768842","Type":"ContainerStarted","Data":"d8869be32695cd436278a8f637bf5e38f2470deface652e6018eecd9e0b5f458"} Sep 18 19:14:20.310280 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.310251 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n5qn8" event={"ID":"d0ae961b-b362-4061-8067-faf028e26649","Type":"ContainerStarted","Data":"dda2bc02ede0703003cd618845cf23458cbbcd1096d8529cf384f0f520f0ce8e"} Sep 18 19:14:20.313594 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.313567 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" event={"ID":"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4","Type":"ContainerStarted","Data":"eb9e1d17762893cd7744cf40591741a6af02a3655be27f6f784dc60ea9562aca"} Sep 18 19:14:20.321358 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.321318 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" event={"ID":"51fae194-f7e3-415c-ad89-4a025211c4b8","Type":"ContainerStarted","Data":"9f82bf4bafbce8d9455f0e642b97acb6e7c6e259adf794f56f48f3ab246f699f"} Sep 18 19:14:20.335582 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.335553 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" event={"ID":"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18","Type":"ContainerStarted","Data":"6a126a6b48a90420bde3a57e00e2f7852ca6481d2f96e1d2eef5cab9c96dcef6"} Sep 18 19:14:20.342024 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.341995 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9fpjr" event={"ID":"bc2c0c9b-3578-46cf-a91e-b3eeef557f64","Type":"ContainerStarted","Data":"42ba7f3116527a721e7d8ea2edaa115d764d3804a969362ef56aa84934a35a82"} Sep 18 19:14:20.351905 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.351850 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" podStartSLOduration=3.351832184 podStartE2EDuration="3.351832184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:20.351742566 +0000 UTC m=+6.256090338" watchObservedRunningTime="2026-09-18 19:14:20.351832184 +0000 UTC m=+6.256179943" Sep 18 19:14:20.355293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.355262 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7kfxm" event={"ID":"d0e6fa6e-811a-4d85-a190-a27c5563dfd9","Type":"ContainerStarted","Data":"b217033f32fe3dacbab376c0338b70af9b3473fee9798c2fb0549160574693e2"} Sep 18 19:14:20.373216 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.373184 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" event={"ID":"484badbc-902c-4577-a5cd-8893ea859178","Type":"ContainerStarted","Data":"e096df6d2578e9c66b8f6cfb11430240426ad514ff250cc1914d506d1c7e4395"} Sep 18 19:14:20.386336 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:20.386306 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-89nfk" event={"ID":"8ed44b0a-93ec-4288-b50b-d0fa7427410c","Type":"ContainerStarted","Data":"3cf694a94a61edff952a445af2d456bd7843fbad26a89f92df27f0e412b89f1d"} Sep 18 19:14:21.405228 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:21.405193 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" event={"ID":"30f01fd6e2b3707a1839770f19ec7ae0","Type":"ContainerStarted","Data":"3184b0662bb70d3c422f181b935e4902692e650e7b667657397a24547488e411"} Sep 18 19:14:21.735585 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:21.735502 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:21.735778 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:21.735694 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:21.735887 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:21.735789 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs podName:0899c67e-8d96-4fa0-b310-b6641c91127b nodeName:}" failed. No retries permitted until 2026-09-18 19:14:25.735741966 +0000 UTC m=+11.640089724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs") pod "network-metrics-daemon-dvkb2" (UID: "0899c67e-8d96-4fa0-b310-b6641c91127b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:21.836098 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:21.836057 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:21.836345 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:21.836325 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:21.836418 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:21.836408 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret podName:d7623d04-8395-4bc7-b628-d82e5e22b847 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:25.83638952 +0000 UTC m=+11.740737258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret") pod "global-pull-secret-syncer-nphvd" (UID: "d7623d04-8395-4bc7-b628-d82e5e22b847") : object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:21.937100 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:21.937064 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swxhx\" (UniqueName: \"kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx\") pod \"network-check-target-2jhvf\" (UID: \"a0a9191c-4b4d-4bfc-8e5d-a2675b620375\") " pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:21.937257 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:21.937237 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 19:14:21.937257 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:21.937256 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 19:14:21.937379 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:21.937270 2554 projected.go:196] Error preparing data for projected volume kube-api-access-swxhx for pod openshift-network-diagnostics/network-check-target-2jhvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:21.937379 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:21.937331 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx podName:a0a9191c-4b4d-4bfc-8e5d-a2675b620375 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:25.937313621 +0000 UTC m=+11.841661361 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-swxhx" (UniqueName: "kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx") pod "network-check-target-2jhvf" (UID: "a0a9191c-4b4d-4bfc-8e5d-a2675b620375") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:22.276661 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:22.276368 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:22.276661 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:22.276492 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:22.277473 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:22.277174 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:22.277473 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:22.277287 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:22.277473 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:22.277346 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:22.277473 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:22.277421 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:24.271821 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:24.271781 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:24.272329 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:24.271935 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:24.272329 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:24.272026 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:24.272329 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:24.272140 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:24.272329 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:24.272189 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:24.272329 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:24.272264 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:25.770268 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:25.770227 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:25.770631 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:25.770375 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:25.770631 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:25.770442 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs podName:0899c67e-8d96-4fa0-b310-b6641c91127b nodeName:}" failed. No retries permitted until 2026-09-18 19:14:33.770421972 +0000 UTC m=+19.674769726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs") pod "network-metrics-daemon-dvkb2" (UID: "0899c67e-8d96-4fa0-b310-b6641c91127b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:25.871032 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:25.870988 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:25.871208 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:25.871181 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:25.871265 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:25.871245 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret podName:d7623d04-8395-4bc7-b628-d82e5e22b847 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:33.871227617 +0000 UTC m=+19.775575364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret") pod "global-pull-secret-syncer-nphvd" (UID: "d7623d04-8395-4bc7-b628-d82e5e22b847") : object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:25.972611 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:25.972128 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swxhx\" (UniqueName: \"kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx\") pod \"network-check-target-2jhvf\" (UID: \"a0a9191c-4b4d-4bfc-8e5d-a2675b620375\") " pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:25.972611 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:25.972333 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 19:14:25.972611 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:25.972360 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 19:14:25.972611 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:25.972372 2554 projected.go:196] Error preparing data for projected volume kube-api-access-swxhx for pod openshift-network-diagnostics/network-check-target-2jhvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:25.972611 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:25.972438 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx podName:a0a9191c-4b4d-4bfc-8e5d-a2675b620375 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:33.972419383 +0000 UTC m=+19.876767119 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-swxhx" (UniqueName: "kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx") pod "network-check-target-2jhvf" (UID: "a0a9191c-4b4d-4bfc-8e5d-a2675b620375") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:26.272411 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:26.272373 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:26.272570 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:26.272470 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:26.272570 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:26.272490 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:26.272570 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:26.272548 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:26.272819 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:26.272594 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:26.272819 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:26.272679 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:28.271613 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:28.271578 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:28.272080 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:28.271577 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:28.272080 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:28.271727 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:28.272080 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:28.271585 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:28.272080 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:28.271792 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:28.272080 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:28.271847 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:30.271088 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:30.271048 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:30.271538 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:30.271057 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:30.271538 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:30.271194 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:30.271538 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:30.271266 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:30.271538 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:30.271058 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:30.271538 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:30.271353 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:32.271437 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:32.271397 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:32.271832 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:32.271397 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:32.271832 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:32.271521 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:32.271832 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:32.271582 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:32.271832 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:32.271397 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:32.271832 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:32.271693 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:33.830911 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:33.830858 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:33.831445 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:33.830985 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:33.831445 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:33.831048 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs podName:0899c67e-8d96-4fa0-b310-b6641c91127b nodeName:}" failed. No retries permitted until 2026-09-18 19:14:49.831033582 +0000 UTC m=+35.735381317 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs") pod "network-metrics-daemon-dvkb2" (UID: "0899c67e-8d96-4fa0-b310-b6641c91127b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:33.931175 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:33.931142 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:33.931345 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:33.931292 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:33.931418 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:33.931358 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret podName:d7623d04-8395-4bc7-b628-d82e5e22b847 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:49.931340899 +0000 UTC m=+35.835688650 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret") pod "global-pull-secret-syncer-nphvd" (UID: "d7623d04-8395-4bc7-b628-d82e5e22b847") : object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:34.032102 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:34.032060 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swxhx\" (UniqueName: \"kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx\") pod \"network-check-target-2jhvf\" (UID: \"a0a9191c-4b4d-4bfc-8e5d-a2675b620375\") " pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:34.032265 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:34.032243 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 19:14:34.032265 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:34.032264 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 19:14:34.032361 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:34.032276 2554 projected.go:196] Error preparing data for projected volume kube-api-access-swxhx for pod openshift-network-diagnostics/network-check-target-2jhvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:34.032361 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:34.032336 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx podName:a0a9191c-4b4d-4bfc-8e5d-a2675b620375 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:50.032321658 +0000 UTC m=+35.936669406 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-swxhx" (UniqueName: "kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx") pod "network-check-target-2jhvf" (UID: "a0a9191c-4b4d-4bfc-8e5d-a2675b620375") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:34.271230 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:34.271143 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:34.271385 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:34.271154 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:34.271385 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:34.271305 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:34.271385 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:34.271155 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:34.271385 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:34.271370 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:34.271581 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:34.271439 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:36.272462 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:36.272432 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:36.272847 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:36.272517 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:36.272847 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:36.272606 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:36.272847 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:36.272723 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:36.272847 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:36.272760 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:36.272847 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:36.272811 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:37.438446 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.438062 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v99gh" event={"ID":"f6f93841-9b28-41ca-aab2-4caefd768842","Type":"ContainerStarted","Data":"413b4f8b890cc3ca99aaf2266be21b2c9f0a17211f0ed17f5ba9d61b0dcf9e71"} Sep 18 19:14:37.441717 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.441553 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n5qn8" event={"ID":"d0ae961b-b362-4061-8067-faf028e26649","Type":"ContainerStarted","Data":"69f03f6c5a4813f6ff09da52a67703e28d58cfbe4968a445cd024c306dc5024c"} Sep 18 19:14:37.446320 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.446298 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:14:37.446728 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.446706 2554 generic.go:356] "Generic (PLEG): container finished" podID="6e2f8a53-2ebd-4d26-9731-0d77648dfcf4" containerID="b97e92c9119b6b5f098313253163deaa786fb83909e6b4b2028ed9739d31c015" exitCode=1 Sep 18 19:14:37.446818 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.446779 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" event={"ID":"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4","Type":"ContainerStarted","Data":"246d4dd420b4856511232e9bffe31ac2248cb227961ad30891c3a9431218b913"} Sep 18 19:14:37.446866 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.446834 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" event={"ID":"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4","Type":"ContainerStarted","Data":"d6ca417fa60ea4708bb32fdbbe680f3e8458302ad6a168ec8b9e0eab42019a3d"} Sep 18 19:14:37.446899 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.446865 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" event={"ID":"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4","Type":"ContainerStarted","Data":"b057b5c6d6973d7f7c23e5be7b12e4aee5edd2c1075e2268bbaf189072b036a8"} Sep 18 19:14:37.446899 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.446894 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" event={"ID":"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4","Type":"ContainerStarted","Data":"7fb37cf77cb4618f24d585f7af25e6d3604bbb80a418d7f355eaed54a8038a4c"} Sep 18 19:14:37.447000 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.446923 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" event={"ID":"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4","Type":"ContainerDied","Data":"b97e92c9119b6b5f098313253163deaa786fb83909e6b4b2028ed9739d31c015"} Sep 18 19:14:37.447000 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.446952 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" event={"ID":"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4","Type":"ContainerStarted","Data":"aee99b6e0f162c7917df1898de838ca0c791b1fbd77039757d9491861b0250eb"} Sep 18 19:14:37.448651 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.448621 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" event={"ID":"51fae194-f7e3-415c-ad89-4a025211c4b8","Type":"ContainerStarted","Data":"084d628815e1d442dd8d3013aa375deefe0221ddad9209adaeb68c0704157b6b"} Sep 18 19:14:37.450504 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.450481 2554 generic.go:356] "Generic (PLEG): container finished" podID="059c4ca0-dcb8-4d39-b3b5-0dfad7944b18" containerID="025b29db42d16f6a5737111fbe88513167054931429bd2cc3afcf218f14d9092" exitCode=0 Sep 18 19:14:37.450596 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.450513 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" event={"ID":"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18","Type":"ContainerDied","Data":"025b29db42d16f6a5737111fbe88513167054931429bd2cc3afcf218f14d9092"} Sep 18 19:14:37.452355 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.452330 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9fpjr" event={"ID":"bc2c0c9b-3578-46cf-a91e-b3eeef557f64","Type":"ContainerStarted","Data":"9688fdee4fc9460949d6fbca0a807c6a574f00995e1a2e24baff5bfe9908c4df"} Sep 18 19:14:37.454063 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.454040 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7kfxm" event={"ID":"d0e6fa6e-811a-4d85-a190-a27c5563dfd9","Type":"ContainerStarted","Data":"05429d3a8d6ee26c93a69a4b209e2ea3793aad1e77e382849949ab4c57d4814d"} Sep 18 19:14:37.455575 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.455554 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" event={"ID":"484badbc-902c-4577-a5cd-8893ea859178","Type":"ContainerStarted","Data":"e46e2200128cf30cf42aea88161e238490f4291404acdb13a3270a909b8ae455"} Sep 18 19:14:37.488002 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.487950 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-v99gh" podStartSLOduration=4.116101633 podStartE2EDuration="21.487936217s" totalImagesPullingTime="17.371834584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:37.484765135 +0000 UTC m=+23.389112911" watchObservedRunningTime="2026-09-18 19:14:37.487936217 +0000 UTC m=+23.392283996" Sep 18 19:14:37.488469 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.488439 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" podStartSLOduration=20.488432223 podStartE2EDuration="20.488432223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:21.476462148 +0000 UTC m=+7.380809933" watchObservedRunningTime="2026-09-18 19:14:37.488432223 +0000 UTC m=+23.392779993" Sep 18 19:14:37.559250 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.559152 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n5qn8" podStartSLOduration=4.12586591 podStartE2EDuration="21.559133611s" totalImagesPullingTime="17.433267701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:37.559086443 +0000 UTC m=+23.463434226" watchObservedRunningTime="2026-09-18 19:14:37.559133611 +0000 UTC m=+23.463481369" Sep 18 19:14:37.559456 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.559428 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7kfxm" podStartSLOduration=8.366365659 podStartE2EDuration="21.559418509s" totalImagesPullingTime="13.19305285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:37.527971603 +0000 UTC m=+23.432319360" watchObservedRunningTime="2026-09-18 19:14:37.559418509 +0000 UTC m=+23.463766269" Sep 18 19:14:37.674975 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.674920 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jrdfc" podStartSLOduration=4.298439331 podStartE2EDuration="21.674902475s" totalImagesPullingTime="17.376463144s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:37.639431695 +0000 UTC m=+23.543779451" watchObservedRunningTime="2026-09-18 19:14:37.674902475 +0000 UTC m=+23.579250231" Sep 18 19:14:37.795948 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.795918 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:37.884619 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:37.884598 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:38.060402 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.060376 2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 18 19:14:38.188496 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.188329 2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T19:14:38.060391436Z","UUID":"149ac9f5-bc01-43f8-a18c-7bf5e965c336","Handler":null,"Name":"","Endpoint":""} Sep 18 19:14:38.189904 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.189884 2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 18 19:14:38.190053 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.189913 2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 18 19:14:38.271421 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.271393 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:38.271596 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.271394 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:38.271596 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:38.271524 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:38.271596 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.271405 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:38.271596 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:38.271586 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:38.271850 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:38.271695 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:38.459957 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.459866 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" event={"ID":"484badbc-902c-4577-a5cd-8893ea859178","Type":"ContainerStarted","Data":"220426ba6cc39987555d451f88e0fe89c37e61fe3ec05d7dc5e9fac840d3635c"} Sep 18 19:14:38.461771 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.461742 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-89nfk" event={"ID":"8ed44b0a-93ec-4288-b50b-d0fa7427410c","Type":"ContainerStarted","Data":"b760d49342becb324e03fb01b35d03ad5e3abc33257235b780d5b3763346ff21"} Sep 18 19:14:38.486857 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.486802 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-89nfk" podStartSLOduration=5.106939445 podStartE2EDuration="22.486784311s" totalImagesPullingTime="17.379844866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:38.486758849 +0000 UTC m=+24.391106632" watchObservedRunningTime="2026-09-18 19:14:38.486784311 +0000 UTC m=+24.391132069" Sep 18 19:14:38.487296 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.487269 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9fpjr" podStartSLOduration=5.12734914 podStartE2EDuration="22.487259445s" totalImagesPullingTime="17.359910305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:37.675049112 +0000 UTC m=+23.579396852" watchObservedRunningTime="2026-09-18 19:14:38.487259445 +0000 UTC m=+24.391607204" Sep 18 19:14:38.753183 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.752996 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:38.920155 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:38.920101 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:39.001267 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:39.001235 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:39.466324 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:39.466228 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" event={"ID":"484badbc-902c-4577-a5cd-8893ea859178","Type":"ContainerStarted","Data":"52e99438216774d2873c17b80a9bf5853b19ca9829c1dbace4efbc0fa7b07423"} Sep 18 19:14:39.493438 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:39.493397 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g5ktj" podStartSLOduration=4.074429856 podStartE2EDuration="23.493381358s" totalImagesPullingTime="19.418951502s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:39.492469095 +0000 UTC m=+25.396816871" watchObservedRunningTime="2026-09-18 19:14:39.493381358 +0000 UTC m=+25.397729118" Sep 18 19:14:40.090529 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:40.090496 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:40.205754 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:40.205718 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:40.271392 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:40.271358 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:40.271392 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:40.271369 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:40.271608 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:40.271477 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:40.271608 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:40.271497 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:40.271724 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:40.271612 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:40.271779 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:40.271745 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:40.279045 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:40.279016 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:40.473095 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:40.472885 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:14:40.473526 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:40.473485 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" event={"ID":"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4","Type":"ContainerStarted","Data":"8a0fe8a891500fb98dba56b302b336390f6499b802ebb6094d1d5057008bbfab"} Sep 18 19:14:41.275694 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:41.275659 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:41.276315 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:41.276297 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:41.924161 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:41.924132 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:42.054149 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.054110 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:42.131746 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.131721 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:42.271689 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.271452 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:42.271850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.271452 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:42.271850 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:42.271703 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:42.271850 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:42.271792 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:42.271850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.271488 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:42.272105 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:42.271900 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:42.481674 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.481628 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:14:42.482050 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.482031 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" event={"ID":"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4","Type":"ContainerStarted","Data":"f0096e75e319e551470366daf37df07040413502dd80ce7703018ff55c3eff2c"} Sep 18 19:14:42.482287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.482263 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:42.482524 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.482500 2554 scope.go:117] "RemoveContainer" containerID="b97e92c9119b6b5f098313253163deaa786fb83909e6b4b2028ed9739d31c015" Sep 18 19:14:42.484072 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.484047 2554 generic.go:356] "Generic (PLEG): container finished" podID="059c4ca0-dcb8-4d39-b3b5-0dfad7944b18" containerID="96b7a864d9ce9f6187c25cdeec25b46c26b4507ad0ec6421234ab391aa7e7b9e" exitCode=0 Sep 18 19:14:42.484144 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.484097 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" event={"ID":"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18","Type":"ContainerDied","Data":"96b7a864d9ce9f6187c25cdeec25b46c26b4507ad0ec6421234ab391aa7e7b9e"} Sep 18 19:14:42.498493 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:42.498471 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:43.490903 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.490873 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:14:43.491784 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.491742 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" event={"ID":"6e2f8a53-2ebd-4d26-9731-0d77648dfcf4","Type":"ContainerStarted","Data":"dcb0c6603f040df70af4308577bbb1e34b831fc4e490180e47ed5bc505b198d5"} Sep 18 19:14:43.491941 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.491914 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:43.492057 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.491963 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:43.507850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.507831 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:14:43.544362 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.544302 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" podStartSLOduration=10.138620905 podStartE2EDuration="27.544280417s" totalImagesPullingTime="17.405659512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:43.542749703 +0000 UTC m=+29.447097485" watchObservedRunningTime="2026-09-18 19:14:43.544280417 +0000 UTC m=+29.448628174" Sep 18 19:14:43.888485 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.888405 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2jhvf"] Sep 18 19:14:43.888681 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.888551 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:43.888681 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:43.888632 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:43.891112 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.891090 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dvkb2"] Sep 18 19:14:43.891235 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.891181 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:43.891276 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:43.891258 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:43.895145 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.895121 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nphvd"] Sep 18 19:14:43.895239 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:43.895229 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:43.895345 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:43.895323 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:44.442219 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:44.442179 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:44.442400 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:44.442329 2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 18 19:14:44.442798 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:44.442774 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7kfxm" Sep 18 19:14:44.496516 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:44.496484 2554 generic.go:356] "Generic (PLEG): container finished" podID="059c4ca0-dcb8-4d39-b3b5-0dfad7944b18" containerID="bc608f075f81f97089f97ec99edd55c01d980b6bdadb6fb043639b57451a90d1" exitCode=0 Sep 18 19:14:44.496961 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:44.496548 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" event={"ID":"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18","Type":"ContainerDied","Data":"bc608f075f81f97089f97ec99edd55c01d980b6bdadb6fb043639b57451a90d1"} Sep 18 19:14:44.529381 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:44.529349 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:44.724176 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:44.724097 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:44.826339 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:44.826304 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:45.271611 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:45.271580 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:45.271786 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:45.271726 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:46.272423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:46.272197 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:46.272816 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:46.272299 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:46.272816 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:46.272500 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:46.272816 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:46.272601 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:46.504037 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:46.503948 2554 generic.go:356] "Generic (PLEG): container finished" podID="059c4ca0-dcb8-4d39-b3b5-0dfad7944b18" containerID="866a4a7ab1be2e0367f952a1f756f63e3ddc7c2ff60c11be4af6fe4f9a5cf1a8" exitCode=0 Sep 18 19:14:46.504037 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:46.504000 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" event={"ID":"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18","Type":"ContainerDied","Data":"866a4a7ab1be2e0367f952a1f756f63e3ddc7c2ff60c11be4af6fe4f9a5cf1a8"} Sep 18 19:14:47.271267 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:47.271236 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:47.271446 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:47.271357 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:48.271692 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:48.271660 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:48.272072 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:48.271662 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:48.272072 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:48.271795 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nphvd" podUID="d7623d04-8395-4bc7-b628-d82e5e22b847" Sep 18 19:14:48.272072 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:48.271844 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:14:49.271090 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.271005 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:49.271318 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:49.271172 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jhvf" podUID="a0a9191c-4b4d-4bfc-8e5d-a2675b620375" Sep 18 19:14:49.475206 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.475178 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeReady" Sep 18 19:14:49.475679 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.475329 2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 18 19:14:49.545988 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.545899 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rbmm8"] Sep 18 19:14:49.569347 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.569322 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n2xhf"] Sep 18 19:14:49.569484 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.569444 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.572289 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.572261 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 18 19:14:49.572550 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.572530 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h7n8q\"" Sep 18 19:14:49.573128 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.573109 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 18 19:14:49.585245 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.585226 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n2xhf" Sep 18 19:14:49.585355 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.585242 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rbmm8"] Sep 18 19:14:49.585355 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.585286 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n2xhf"] Sep 18 19:14:49.588897 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.588876 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 18 19:14:49.589014 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.588899 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 18 19:14:49.589014 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.588911 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 18 19:14:49.589615 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.589596 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lcfbs\"" Sep 18 19:14:49.639658 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.639606 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9996e7d0-59d0-4457-8967-9a4c7eb67967-tmp-dir\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.639833 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.639666 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnljj\" (UniqueName: \"kubernetes.io/projected/9996e7d0-59d0-4457-8967-9a4c7eb67967-kube-api-access-mnljj\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.639833 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.639739 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9996e7d0-59d0-4457-8967-9a4c7eb67967-config-volume\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.639833 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.639775 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.740907 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.740878 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9996e7d0-59d0-4457-8967-9a4c7eb67967-tmp-dir\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.741115 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.740913 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/879e230b-7166-438e-b7d4-e88e9116a54b-cert\") pod \"ingress-canary-n2xhf\" (UID: \"879e230b-7166-438e-b7d4-e88e9116a54b\") " pod="openshift-ingress-canary/ingress-canary-n2xhf" Sep 18 19:14:49.741115 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.740930 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8567s\" (UniqueName: \"kubernetes.io/projected/879e230b-7166-438e-b7d4-e88e9116a54b-kube-api-access-8567s\") pod \"ingress-canary-n2xhf\" (UID: \"879e230b-7166-438e-b7d4-e88e9116a54b\") " pod="openshift-ingress-canary/ingress-canary-n2xhf" Sep 18 19:14:49.741115 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.740955 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnljj\" (UniqueName: \"kubernetes.io/projected/9996e7d0-59d0-4457-8967-9a4c7eb67967-kube-api-access-mnljj\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.741115 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.741044 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9996e7d0-59d0-4457-8967-9a4c7eb67967-config-volume\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.741115 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.741098 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.741377 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:49.741189 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 19:14:49.741377 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:49.741235 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls podName:9996e7d0-59d0-4457-8967-9a4c7eb67967 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:50.241222145 +0000 UTC m=+36.145569893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls") pod "dns-default-rbmm8" (UID: "9996e7d0-59d0-4457-8967-9a4c7eb67967") : secret "dns-default-metrics-tls" not found Sep 18 19:14:49.741377 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.741232 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9996e7d0-59d0-4457-8967-9a4c7eb67967-tmp-dir\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.741707 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.741686 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9996e7d0-59d0-4457-8967-9a4c7eb67967-config-volume\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.750534 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.750507 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:49.758319 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.758290 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnljj\" (UniqueName: \"kubernetes.io/projected/9996e7d0-59d0-4457-8967-9a4c7eb67967-kube-api-access-mnljj\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:49.841947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.841861 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:49.842088 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.841952 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/879e230b-7166-438e-b7d4-e88e9116a54b-cert\") pod \"ingress-canary-n2xhf\" (UID: \"879e230b-7166-438e-b7d4-e88e9116a54b\") " pod="openshift-ingress-canary/ingress-canary-n2xhf" Sep 18 19:14:49.842088 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.841976 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8567s\" (UniqueName: \"kubernetes.io/projected/879e230b-7166-438e-b7d4-e88e9116a54b-kube-api-access-8567s\") pod \"ingress-canary-n2xhf\" (UID: \"879e230b-7166-438e-b7d4-e88e9116a54b\") " pod="openshift-ingress-canary/ingress-canary-n2xhf" Sep 18 19:14:49.842088 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:49.842049 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:49.842227 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:49.842118 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs podName:0899c67e-8d96-4fa0-b310-b6641c91127b nodeName:}" failed. No retries permitted until 2026-09-18 19:15:21.842099968 +0000 UTC m=+67.746447717 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs") pod "network-metrics-daemon-dvkb2" (UID: "0899c67e-8d96-4fa0-b310-b6641c91127b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 19:14:49.843322 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.843296 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:49.844909 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.844886 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/879e230b-7166-438e-b7d4-e88e9116a54b-cert\") pod \"ingress-canary-n2xhf\" (UID: \"879e230b-7166-438e-b7d4-e88e9116a54b\") " pod="openshift-ingress-canary/ingress-canary-n2xhf" Sep 18 19:14:49.858952 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.858923 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8567s\" (UniqueName: \"kubernetes.io/projected/879e230b-7166-438e-b7d4-e88e9116a54b-kube-api-access-8567s\") pod \"ingress-canary-n2xhf\" (UID: \"879e230b-7166-438e-b7d4-e88e9116a54b\") " pod="openshift-ingress-canary/ingress-canary-n2xhf" Sep 18 19:14:49.894880 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.894845 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n2xhf" Sep 18 19:14:49.942549 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:49.942513 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:49.942717 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:49.942696 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:49.942798 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:49.942786 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret podName:d7623d04-8395-4bc7-b628-d82e5e22b847 nodeName:}" failed. No retries permitted until 2026-09-18 19:15:21.942764897 +0000 UTC m=+67.847112635 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret") pod "global-pull-secret-syncer-nphvd" (UID: "d7623d04-8395-4bc7-b628-d82e5e22b847") : object "kube-system"/"original-pull-secret" not registered Sep 18 19:14:50.043258 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:50.043221 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swxhx\" (UniqueName: \"kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx\") pod \"network-check-target-2jhvf\" (UID: \"a0a9191c-4b4d-4bfc-8e5d-a2675b620375\") " pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:50.043444 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:50.043382 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 19:14:50.043444 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:50.043411 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 19:14:50.043444 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:50.043424 2554 projected.go:196] Error preparing data for projected volume kube-api-access-swxhx for pod openshift-network-diagnostics/network-check-target-2jhvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:50.043566 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:50.043492 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx podName:a0a9191c-4b4d-4bfc-8e5d-a2675b620375 nodeName:}" failed. No retries permitted until 2026-09-18 19:15:22.043473659 +0000 UTC m=+67.947821419 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-swxhx" (UniqueName: "kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx") pod "network-check-target-2jhvf" (UID: "a0a9191c-4b4d-4bfc-8e5d-a2675b620375") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 19:14:50.245359 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:50.245321 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:50.245556 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:50.245453 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 19:14:50.245556 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:50.245511 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls podName:9996e7d0-59d0-4457-8967-9a4c7eb67967 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:51.24549765 +0000 UTC m=+37.149845386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls") pod "dns-default-rbmm8" (UID: "9996e7d0-59d0-4457-8967-9a4c7eb67967") : secret "dns-default-metrics-tls" not found Sep 18 19:14:50.271309 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:50.271277 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:14:50.271497 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:50.271474 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:14:50.274003 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:50.273983 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 19:14:50.274695 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:50.274679 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w2qlj\"" Sep 18 19:14:50.274917 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:50.274905 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 19:14:51.254467 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:51.254400 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:51.255034 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:51.254552 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 19:14:51.255034 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:51.254613 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls podName:9996e7d0-59d0-4457-8967-9a4c7eb67967 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:53.254598293 +0000 UTC m=+39.158946051 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls") pod "dns-default-rbmm8" (UID: "9996e7d0-59d0-4457-8967-9a4c7eb67967") : secret "dns-default-metrics-tls" not found Sep 18 19:14:51.272084 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:51.271932 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:14:51.276465 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:51.276447 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 19:14:51.279024 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:51.279008 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 19:14:51.279558 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:51.279542 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5n7ds\"" Sep 18 19:14:51.294115 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:51.293343 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n2xhf"] Sep 18 19:14:51.301451 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:14:51.301427 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod879e230b_7166_438e_b7d4_e88e9116a54b.slice/crio-4e7db61c83cb1dcc3569d13785fd74387b38c7fc25402f1aba43104f37bce16b WatchSource:0}: Error finding container 4e7db61c83cb1dcc3569d13785fd74387b38c7fc25402f1aba43104f37bce16b: Status 404 returned error can't find the container with id 4e7db61c83cb1dcc3569d13785fd74387b38c7fc25402f1aba43104f37bce16b Sep 18 19:14:51.519764 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:51.519654 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" event={"ID":"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18","Type":"ContainerStarted","Data":"acab10eac51078c75083d87d41c05085ba5cbcb24f1d747384190df8015bdb27"} Sep 18 19:14:51.521003 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:51.520962 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n2xhf" event={"ID":"879e230b-7166-438e-b7d4-e88e9116a54b","Type":"ContainerStarted","Data":"4e7db61c83cb1dcc3569d13785fd74387b38c7fc25402f1aba43104f37bce16b"} Sep 18 19:14:52.457007 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:52.456958 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:52.527365 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:52.527324 2554 generic.go:356] "Generic (PLEG): container finished" podID="059c4ca0-dcb8-4d39-b3b5-0dfad7944b18" containerID="acab10eac51078c75083d87d41c05085ba5cbcb24f1d747384190df8015bdb27" exitCode=0 Sep 18 19:14:52.527525 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:52.527392 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" event={"ID":"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18","Type":"ContainerDied","Data":"acab10eac51078c75083d87d41c05085ba5cbcb24f1d747384190df8015bdb27"} Sep 18 19:14:53.270654 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:53.270600 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:53.270834 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:53.270762 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 19:14:53.270896 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:53.270835 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls podName:9996e7d0-59d0-4457-8967-9a4c7eb67967 nodeName:}" failed. No retries permitted until 2026-09-18 19:14:57.270814891 +0000 UTC m=+43.175162640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls") pod "dns-default-rbmm8" (UID: "9996e7d0-59d0-4457-8967-9a4c7eb67967") : secret "dns-default-metrics-tls" not found Sep 18 19:14:53.532307 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:53.532234 2554 generic.go:356] "Generic (PLEG): container finished" podID="059c4ca0-dcb8-4d39-b3b5-0dfad7944b18" containerID="d1b7414b51864f164fe7122b790905821898ec873cbcfba515d568c576226d79" exitCode=0 Sep 18 19:14:53.532307 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:53.532275 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" event={"ID":"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18","Type":"ContainerDied","Data":"d1b7414b51864f164fe7122b790905821898ec873cbcfba515d568c576226d79"} Sep 18 19:14:54.539562 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:54.539324 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n2xhf" event={"ID":"879e230b-7166-438e-b7d4-e88e9116a54b","Type":"ContainerStarted","Data":"530cc73eb530885a9098d5bd8b9e947b5f8cdead4dffd7af6e98c7a67b6a7a1a"} Sep 18 19:14:54.542845 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:54.542819 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" event={"ID":"059c4ca0-dcb8-4d39-b3b5-0dfad7944b18","Type":"ContainerStarted","Data":"8f7532d19bd7af72ba5988681c4240a8f25d5874cc13474d15ee61498372b046"} Sep 18 19:14:54.619448 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:54.619392 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qrv5j" podStartSLOduration=10.001198832 podStartE2EDuration="38.619375309s" totalImagesPullingTime="28.618176477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:54.617575499 +0000 UTC m=+40.521923275" watchObservedRunningTime="2026-09-18 19:14:54.619375309 +0000 UTC m=+40.523723066" Sep 18 19:14:54.619679 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:54.619618 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n2xhf" podStartSLOduration=3.118159395 podStartE2EDuration="5.619611258s" totalImagesPullingTime="2.501451863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:14:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:54.573831075 +0000 UTC m=+40.478178832" watchObservedRunningTime="2026-09-18 19:14:54.619611258 +0000 UTC m=+40.523959015" Sep 18 19:14:54.766996 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:54.766950 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:54.857003 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:54.856919 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:54.959410 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:54.959377 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:55.068792 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:55.068747 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:55.172128 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:55.172083 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:56.581382 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:56.581349 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:56.887961 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:56.887930 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:57.033020 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:57.032990 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:57.137093 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:57.137066 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:57.204100 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:57.204030 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:57.299630 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:57.299596 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:14:57.299785 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:57.299725 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 19:14:57.299820 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:14:57.299785 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls podName:9996e7d0-59d0-4457-8967-9a4c7eb67967 nodeName:}" failed. No retries permitted until 2026-09-18 19:15:05.299768973 +0000 UTC m=+51.204116722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls") pod "dns-default-rbmm8" (UID: "9996e7d0-59d0-4457-8967-9a4c7eb67967") : secret "dns-default-metrics-tls" not found Sep 18 19:14:57.763280 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:57.763254 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:14:57.841353 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:14:57.841330 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:15:05.354959 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:05.354915 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:15:05.355421 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:15:05.355063 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 19:15:05.355421 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:15:05.355128 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls podName:9996e7d0-59d0-4457-8967-9a4c7eb67967 nodeName:}" failed. No retries permitted until 2026-09-18 19:15:21.355113988 +0000 UTC m=+67.259461723 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls") pod "dns-default-rbmm8" (UID: "9996e7d0-59d0-4457-8967-9a4c7eb67967") : secret "dns-default-metrics-tls" not found Sep 18 19:15:15.514279 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:15.514251 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqqtf" Sep 18 19:15:15.685335 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:15.685273 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:15:15.758961 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:15.758876 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:15:15.828013 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:15.827914 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:15:18.848984 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:18.848951 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:15:18.945194 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:18.945152 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:15:19.009957 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:19.009916 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:15:21.374086 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:21.374049 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:15:21.374467 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:15:21.374207 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 19:15:21.374467 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:15:21.374284 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls podName:9996e7d0-59d0-4457-8967-9a4c7eb67967 nodeName:}" failed. No retries permitted until 2026-09-18 19:15:53.374268064 +0000 UTC m=+99.278615799 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls") pod "dns-default-rbmm8" (UID: "9996e7d0-59d0-4457-8967-9a4c7eb67967") : secret "dns-default-metrics-tls" not found Sep 18 19:15:21.878325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:21.878277 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:15:21.880881 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:21.880864 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 19:15:21.888919 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:15:21.888902 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 18 19:15:21.888992 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:15:21.888982 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs podName:0899c67e-8d96-4fa0-b310-b6641c91127b nodeName:}" failed. No retries permitted until 2026-09-18 19:16:25.888967541 +0000 UTC m=+131.793315291 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs") pod "network-metrics-daemon-dvkb2" (UID: "0899c67e-8d96-4fa0-b310-b6641c91127b") : secret "metrics-daemon-secret" not found Sep 18 19:15:21.979582 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:21.979537 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:15:21.982353 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:21.982337 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 19:15:21.993521 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:21.993492 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7623d04-8395-4bc7-b628-d82e5e22b847-original-pull-secret\") pod \"global-pull-secret-syncer-nphvd\" (UID: \"d7623d04-8395-4bc7-b628-d82e5e22b847\") " pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:15:22.080045 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:22.079993 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swxhx\" (UniqueName: \"kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx\") pod \"network-check-target-2jhvf\" (UID: \"a0a9191c-4b4d-4bfc-8e5d-a2675b620375\") " pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:15:22.082728 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:22.082707 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 19:15:22.087697 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:22.087680 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nphvd" Sep 18 19:15:22.092626 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:22.092609 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 19:15:22.103728 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:22.103704 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swxhx\" (UniqueName: \"kubernetes.io/projected/a0a9191c-4b4d-4bfc-8e5d-a2675b620375-kube-api-access-swxhx\") pod \"network-check-target-2jhvf\" (UID: \"a0a9191c-4b4d-4bfc-8e5d-a2675b620375\") " pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:15:22.183774 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:22.183745 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5n7ds\"" Sep 18 19:15:22.191733 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:22.191697 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:15:22.356969 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:22.356937 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nphvd"] Sep 18 19:15:22.359585 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:15:22.359566 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7623d04_8395_4bc7_b628_d82e5e22b847.slice/crio-da11b5494b716dc7bc166788b7ad78f4e22e659d8b5f854a9e6d21eda3fa525c WatchSource:0}: Error finding container da11b5494b716dc7bc166788b7ad78f4e22e659d8b5f854a9e6d21eda3fa525c: Status 404 returned error can't find the container with id da11b5494b716dc7bc166788b7ad78f4e22e659d8b5f854a9e6d21eda3fa525c Sep 18 19:15:22.469326 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:22.469257 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2jhvf"] Sep 18 19:15:22.471886 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:15:22.471844 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a9191c_4b4d_4bfc_8e5d_a2675b620375.slice/crio-2acaca8f8d214370b7abdff10b5b3ea7f086ced388789f8617478cc1698b2c42 WatchSource:0}: Error finding container 2acaca8f8d214370b7abdff10b5b3ea7f086ced388789f8617478cc1698b2c42: Status 404 returned error can't find the container with id 2acaca8f8d214370b7abdff10b5b3ea7f086ced388789f8617478cc1698b2c42 Sep 18 19:15:22.612729 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:22.612683 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2jhvf" event={"ID":"a0a9191c-4b4d-4bfc-8e5d-a2675b620375","Type":"ContainerStarted","Data":"2acaca8f8d214370b7abdff10b5b3ea7f086ced388789f8617478cc1698b2c42"} Sep 18 19:15:22.613622 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:22.613599 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nphvd" event={"ID":"d7623d04-8395-4bc7-b628-d82e5e22b847","Type":"ContainerStarted","Data":"da11b5494b716dc7bc166788b7ad78f4e22e659d8b5f854a9e6d21eda3fa525c"} Sep 18 19:15:27.629015 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:27.628873 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2jhvf" event={"ID":"a0a9191c-4b4d-4bfc-8e5d-a2675b620375","Type":"ContainerStarted","Data":"87e3382e61e4ca8a8437e34f5c7f0cd1d50ad83434a304cc5cdc474beb319aa9"} Sep 18 19:15:27.629015 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:27.629008 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:15:27.630360 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:27.630338 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nphvd" event={"ID":"d7623d04-8395-4bc7-b628-d82e5e22b847","Type":"ContainerStarted","Data":"6eed40d33b72570d40712be752852525b3b5d2a5d45ee25b6947c27bf97069ad"} Sep 18 19:15:27.646918 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:27.646870 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2jhvf" podStartSLOduration=67.400908699 podStartE2EDuration="1m11.646853711s" totalImagesPullingTime="4.245945012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:15:27.645549209 +0000 UTC m=+73.549896981" watchObservedRunningTime="2026-09-18 19:15:27.646853711 +0000 UTC m=+73.551201467" Sep 18 19:15:27.661539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:27.661481 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nphvd" podStartSLOduration=67.300279478 podStartE2EDuration="1m11.661463404s" totalImagesPullingTime="4.361183926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:15:27.660772981 +0000 UTC m=+73.565120760" watchObservedRunningTime="2026-09-18 19:15:27.661463404 +0000 UTC m=+73.565811154" Sep 18 19:15:53.386704 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:53.386629 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:15:53.387295 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:15:53.386779 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 19:15:53.387295 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:15:53.386855 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls podName:9996e7d0-59d0-4457-8967-9a4c7eb67967 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:57.386834581 +0000 UTC m=+163.291182338 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls") pod "dns-default-rbmm8" (UID: "9996e7d0-59d0-4457-8967-9a4c7eb67967") : secret "dns-default-metrics-tls" not found Sep 18 19:15:58.635109 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:15:58.635075 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2jhvf" Sep 18 19:16:09.581280 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.581247 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-54788fddc8-7b577"] Sep 18 19:16:09.583470 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.583454 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.586823 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.586803 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 18 19:16:09.587079 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.587061 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 18 19:16:09.587142 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.587129 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-27vxx\"" Sep 18 19:16:09.587373 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.587353 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 18 19:16:09.587463 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.587352 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 18 19:16:09.587948 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.587934 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 18 19:16:09.588237 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.588222 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 18 19:16:09.617795 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.617768 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54788fddc8-7b577"] Sep 18 19:16:09.695114 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.695080 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg58c\" (UniqueName: \"kubernetes.io/projected/2cee06b4-79a2-47d4-9443-e1d1353ccc82-kube-api-access-wg58c\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.695114 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.695115 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-stats-auth\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.695320 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.695141 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.695320 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.695198 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.695320 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.695246 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-default-certificate\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.777233 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.777200 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ghj6f"] Sep 18 19:16:09.779288 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.779270 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ghj6f" Sep 18 19:16:09.782000 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.781981 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 18 19:16:09.782624 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.782610 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 18 19:16:09.782840 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.782827 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tnz4z\"" Sep 18 19:16:09.793652 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.793606 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl"] Sep 18 19:16:09.795443 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.795422 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:09.795612 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.795590 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg58c\" (UniqueName: \"kubernetes.io/projected/2cee06b4-79a2-47d4-9443-e1d1353ccc82-kube-api-access-wg58c\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.795798 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.795772 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-stats-auth\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.795887 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.795829 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.795887 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.795855 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.795996 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.795888 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-default-certificate\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.795996 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:09.795935 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 19:16:09.796089 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:09.796006 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:10.295987432 +0000 UTC m=+116.200335182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : secret "router-metrics-certs-default" not found Sep 18 19:16:09.796150 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:09.796120 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:10.296102357 +0000 UTC m=+116.200450109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : configmap references non-existent config key: service-ca.crt Sep 18 19:16:09.798491 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.798467 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-stats-auth\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.798590 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.798515 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-default-certificate\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.799623 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.799605 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 18 19:16:09.803596 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.803575 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 18 19:16:09.803883 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.803866 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 18 19:16:09.803958 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.803905 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 18 19:16:09.819400 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.819372 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ghj6f"] Sep 18 19:16:09.823615 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.823588 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl"] Sep 18 19:16:09.828911 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.828887 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg58c\" (UniqueName: \"kubernetes.io/projected/2cee06b4-79a2-47d4-9443-e1d1353ccc82-kube-api-access-wg58c\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:09.897133 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.897094 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbz2s\" (UniqueName: \"kubernetes.io/projected/08f56945-5dc0-4066-b87a-b406e99bd2f0-kube-api-access-hbz2s\") pod \"klusterlet-addon-workmgr-85ffd8c7f7-fklcl\" (UID: \"08f56945-5dc0-4066-b87a-b406e99bd2f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:09.897310 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.897190 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/08f56945-5dc0-4066-b87a-b406e99bd2f0-klusterlet-config\") pod \"klusterlet-addon-workmgr-85ffd8c7f7-fklcl\" (UID: \"08f56945-5dc0-4066-b87a-b406e99bd2f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:09.897310 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.897241 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f56945-5dc0-4066-b87a-b406e99bd2f0-tmp\") pod \"klusterlet-addon-workmgr-85ffd8c7f7-fklcl\" (UID: \"08f56945-5dc0-4066-b87a-b406e99bd2f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:09.897310 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.897267 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9djc\" (UniqueName: \"kubernetes.io/projected/c88b76e1-4c65-4f0c-aa2f-00cc0419a7e3-kube-api-access-w9djc\") pod \"volume-data-source-validator-5c4db8bdf9-ghj6f\" (UID: \"c88b76e1-4c65-4f0c-aa2f-00cc0419a7e3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ghj6f" Sep 18 19:16:09.920887 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.920860 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-598d87595f-rn9x8"] Sep 18 19:16:09.922814 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.922796 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:09.930580 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.930559 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 18 19:16:09.930706 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.930583 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qbqcn\"" Sep 18 19:16:09.931215 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.931199 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 18 19:16:09.932778 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.932763 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 18 19:16:09.941493 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.941469 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 18 19:16:09.954141 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.954118 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-598d87595f-rn9x8"] Sep 18 19:16:09.997872 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.997839 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-installation-pull-secrets\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:09.997872 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.997874 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-certificates\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:09.998089 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.997900 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/08f56945-5dc0-4066-b87a-b406e99bd2f0-klusterlet-config\") pod \"klusterlet-addon-workmgr-85ffd8c7f7-fklcl\" (UID: \"08f56945-5dc0-4066-b87a-b406e99bd2f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:09.998089 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.997919 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-bound-sa-token\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:09.998089 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.997977 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:09.998089 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.998011 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8a7681ba-500b-466c-9094-0d160a51d4fa-ca-trust-extracted\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:09.998089 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.998033 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f56945-5dc0-4066-b87a-b406e99bd2f0-tmp\") pod \"klusterlet-addon-workmgr-85ffd8c7f7-fklcl\" (UID: \"08f56945-5dc0-4066-b87a-b406e99bd2f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:09.998089 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.998048 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-trusted-ca\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:09.998089 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.998065 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tld5f\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-kube-api-access-tld5f\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:09.998415 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.998146 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9djc\" (UniqueName: \"kubernetes.io/projected/c88b76e1-4c65-4f0c-aa2f-00cc0419a7e3-kube-api-access-w9djc\") pod \"volume-data-source-validator-5c4db8bdf9-ghj6f\" (UID: \"c88b76e1-4c65-4f0c-aa2f-00cc0419a7e3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ghj6f" Sep 18 19:16:09.998415 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.998210 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-image-registry-private-configuration\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:09.998415 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.998233 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbz2s\" (UniqueName: \"kubernetes.io/projected/08f56945-5dc0-4066-b87a-b406e99bd2f0-kube-api-access-hbz2s\") pod \"klusterlet-addon-workmgr-85ffd8c7f7-fklcl\" (UID: \"08f56945-5dc0-4066-b87a-b406e99bd2f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:09.998415 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:09.998396 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f56945-5dc0-4066-b87a-b406e99bd2f0-tmp\") pod \"klusterlet-addon-workmgr-85ffd8c7f7-fklcl\" (UID: \"08f56945-5dc0-4066-b87a-b406e99bd2f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:10.000381 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.000361 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/08f56945-5dc0-4066-b87a-b406e99bd2f0-klusterlet-config\") pod \"klusterlet-addon-workmgr-85ffd8c7f7-fklcl\" (UID: \"08f56945-5dc0-4066-b87a-b406e99bd2f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:10.035853 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.035823 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:10.051105 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.051080 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9djc\" (UniqueName: \"kubernetes.io/projected/c88b76e1-4c65-4f0c-aa2f-00cc0419a7e3-kube-api-access-w9djc\") pod \"volume-data-source-validator-5c4db8bdf9-ghj6f\" (UID: \"c88b76e1-4c65-4f0c-aa2f-00cc0419a7e3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ghj6f" Sep 18 19:16:10.075952 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.075921 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbz2s\" (UniqueName: \"kubernetes.io/projected/08f56945-5dc0-4066-b87a-b406e99bd2f0-kube-api-access-hbz2s\") pod \"klusterlet-addon-workmgr-85ffd8c7f7-fklcl\" (UID: \"08f56945-5dc0-4066-b87a-b406e99bd2f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:10.087719 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.087697 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ghj6f" Sep 18 19:16:10.099426 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.099407 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-trusted-ca\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.099517 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.099433 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tld5f\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-kube-api-access-tld5f\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.099517 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.099472 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-image-registry-private-configuration\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.099683 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.099667 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-installation-pull-secrets\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.099740 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.099698 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-certificates\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.099740 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.099734 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-bound-sa-token\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.099840 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.099780 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.099840 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.099809 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8a7681ba-500b-466c-9094-0d160a51d4fa-ca-trust-extracted\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.100052 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:10.100028 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 19:16:10.100130 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:10.100056 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598d87595f-rn9x8: secret "image-registry-tls" not found Sep 18 19:16:10.100130 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:10.100123 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls podName:8a7681ba-500b-466c-9094-0d160a51d4fa nodeName:}" failed. No retries permitted until 2026-09-18 19:16:10.600103203 +0000 UTC m=+116.504450952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls") pod "image-registry-598d87595f-rn9x8" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa") : secret "image-registry-tls" not found Sep 18 19:16:10.100246 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.100137 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8a7681ba-500b-466c-9094-0d160a51d4fa-ca-trust-extracted\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.100300 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.100279 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-certificates\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.100468 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.100446 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-trusted-ca\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.101815 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.101796 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-image-registry-private-configuration\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.102207 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.102188 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-installation-pull-secrets\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.109112 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.109090 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:10.141224 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.141089 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-bound-sa-token\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.141224 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.141171 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tld5f\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-kube-api-access-tld5f\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.301549 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.301514 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:10.301549 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.301549 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:10.301772 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:10.301685 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 19:16:10.301772 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:10.301717 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:11.301702855 +0000 UTC m=+117.206050603 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : configmap references non-existent config key: service-ca.crt Sep 18 19:16:10.301772 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:10.301739 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:11.301725077 +0000 UTC m=+117.206072811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : secret "router-metrics-certs-default" not found Sep 18 19:16:10.369485 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.369456 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ghj6f"] Sep 18 19:16:10.373473 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:10.373448 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc88b76e1_4c65_4f0c_aa2f_00cc0419a7e3.slice/crio-522d08d214b461c5d91f7b3557c138a8420d6ecc741f49e2fcaff367db2e9105 WatchSource:0}: Error finding container 522d08d214b461c5d91f7b3557c138a8420d6ecc741f49e2fcaff367db2e9105: Status 404 returned error can't find the container with id 522d08d214b461c5d91f7b3557c138a8420d6ecc741f49e2fcaff367db2e9105 Sep 18 19:16:10.399946 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.399921 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl"] Sep 18 19:16:10.403390 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:10.403365 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08f56945_5dc0_4066_b87a_b406e99bd2f0.slice/crio-19e1524913c3f089c9b4bca614946bafe3876667c0632f5d6f69052ff7cb019c WatchSource:0}: Error finding container 19e1524913c3f089c9b4bca614946bafe3876667c0632f5d6f69052ff7cb019c: Status 404 returned error can't find the container with id 19e1524913c3f089c9b4bca614946bafe3876667c0632f5d6f69052ff7cb019c Sep 18 19:16:10.603893 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.603812 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:10.604256 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:10.603923 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 19:16:10.604256 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:10.603935 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598d87595f-rn9x8: secret "image-registry-tls" not found Sep 18 19:16:10.604256 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:10.603987 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls podName:8a7681ba-500b-466c-9094-0d160a51d4fa nodeName:}" failed. No retries permitted until 2026-09-18 19:16:11.603971927 +0000 UTC m=+117.508319675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls") pod "image-registry-598d87595f-rn9x8" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa") : secret "image-registry-tls" not found Sep 18 19:16:10.738943 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.738908 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" event={"ID":"08f56945-5dc0-4066-b87a-b406e99bd2f0","Type":"ContainerStarted","Data":"19e1524913c3f089c9b4bca614946bafe3876667c0632f5d6f69052ff7cb019c"} Sep 18 19:16:10.739953 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:10.739932 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ghj6f" event={"ID":"c88b76e1-4c65-4f0c-aa2f-00cc0419a7e3","Type":"ContainerStarted","Data":"522d08d214b461c5d91f7b3557c138a8420d6ecc741f49e2fcaff367db2e9105"} Sep 18 19:16:11.257023 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:11.256985 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:11.309613 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:11.309577 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:11.309613 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:11.309617 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:11.309813 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:11.309773 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:13.30975874 +0000 UTC m=+119.214106488 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : configmap references non-existent config key: service-ca.crt Sep 18 19:16:11.309960 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:11.309939 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 19:16:11.310008 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:11.309999 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:13.309985068 +0000 UTC m=+119.214332824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : secret "router-metrics-certs-default" not found Sep 18 19:16:11.612700 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:11.612611 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:11.613126 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:11.612782 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 19:16:11.613126 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:11.612811 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598d87595f-rn9x8: secret "image-registry-tls" not found Sep 18 19:16:11.613126 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:11.612876 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls podName:8a7681ba-500b-466c-9094-0d160a51d4fa nodeName:}" failed. No retries permitted until 2026-09-18 19:16:13.61286065 +0000 UTC m=+119.517208403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls") pod "image-registry-598d87595f-rn9x8" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa") : secret "image-registry-tls" not found Sep 18 19:16:11.745158 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:11.745038 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ghj6f" event={"ID":"c88b76e1-4c65-4f0c-aa2f-00cc0419a7e3","Type":"ContainerStarted","Data":"1907510aa8be772276696c498d3f968e2d5fd54ff0439388515df1ef0e9c98b1"} Sep 18 19:16:11.877799 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:11.877710 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:11.978493 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:11.978459 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:13.327556 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:13.327518 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:13.327556 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:13.327556 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:13.328030 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:13.327684 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 19:16:13.328030 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:13.327737 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:17.327713128 +0000 UTC m=+123.232060880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : configmap references non-existent config key: service-ca.crt Sep 18 19:16:13.328030 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:13.327766 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:17.327755799 +0000 UTC m=+123.232103542 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : secret "router-metrics-certs-default" not found Sep 18 19:16:13.629723 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:13.629627 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:13.629874 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:13.629782 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 19:16:13.629874 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:13.629807 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598d87595f-rn9x8: secret "image-registry-tls" not found Sep 18 19:16:13.629987 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:13.629874 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls podName:8a7681ba-500b-466c-9094-0d160a51d4fa nodeName:}" failed. No retries permitted until 2026-09-18 19:16:17.629853362 +0000 UTC m=+123.534201099 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls") pod "image-registry-598d87595f-rn9x8" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa") : secret "image-registry-tls" not found Sep 18 19:16:14.755335 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:14.755297 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" event={"ID":"08f56945-5dc0-4066-b87a-b406e99bd2f0","Type":"ContainerStarted","Data":"f6941ce1e5deb457fc22e4bd6a188c9f1acfd62c7356b049fc4d05bd1d6dc3a4"} Sep 18 19:16:14.755823 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:14.755498 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:14.757075 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:14.757057 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" Sep 18 19:16:14.784676 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:14.784620 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ghj6f" podStartSLOduration=4.833242535 podStartE2EDuration="5.7846082s" totalImagesPullingTime="951.365665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:11.768861722 +0000 UTC m=+117.673209509" watchObservedRunningTime="2026-09-18 19:16:14.7846082 +0000 UTC m=+120.688955951" Sep 18 19:16:14.784921 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:14.784901 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffd8c7f7-fklcl" podStartSLOduration=2.029425884 podStartE2EDuration="5.784894045s" totalImagesPullingTime="3.755468161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:14.784751001 +0000 UTC m=+120.689098745" watchObservedRunningTime="2026-09-18 19:16:14.784894045 +0000 UTC m=+120.689241876" Sep 18 19:16:15.435872 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.435842 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wqxts"] Sep 18 19:16:15.437504 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.437479 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:15.437837 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.437823 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.456683 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.456664 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 18 19:16:15.456807 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.456758 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-npv6k\"" Sep 18 19:16:15.456889 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.456874 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 18 19:16:15.456941 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.456874 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 18 19:16:15.456994 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.456949 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 18 19:16:15.458051 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.458029 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wqxts"] Sep 18 19:16:15.545895 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.545857 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.546061 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.545933 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/15d1835a-24a8-4c8b-9c73-4afba0b63366-data-volume\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.546061 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.545960 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98cdb\" (UniqueName: \"kubernetes.io/projected/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-api-access-98cdb\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.546061 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.545993 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.546169 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.546074 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/15d1835a-24a8-4c8b-9c73-4afba0b63366-crio-socket\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.556197 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.556164 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:15.646784 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.646741 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.646902 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:15.646892 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 19:16:15.646950 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.646901 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/15d1835a-24a8-4c8b-9c73-4afba0b63366-crio-socket\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.646950 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.646931 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.647043 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:15.646955 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls podName:15d1835a-24a8-4c8b-9c73-4afba0b63366 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:16.146940779 +0000 UTC m=+122.051288514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqxts" (UID: "15d1835a-24a8-4c8b-9c73-4afba0b63366") : secret "insights-runtime-extractor-tls" not found Sep 18 19:16:15.647043 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:15.646979 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 19:16:15.647043 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.647001 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/15d1835a-24a8-4c8b-9c73-4afba0b63366-data-volume\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.647043 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.647011 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/15d1835a-24a8-4c8b-9c73-4afba0b63366-crio-socket\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.647043 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.647022 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98cdb\" (UniqueName: \"kubernetes.io/projected/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-api-access-98cdb\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.647043 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:15.647035 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm podName:15d1835a-24a8-4c8b-9c73-4afba0b63366 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:16.14701879 +0000 UTC m=+122.051366545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqxts" (UID: "15d1835a-24a8-4c8b-9c73-4afba0b63366") : configmap "kube-rbac-proxy" not found Sep 18 19:16:15.647356 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.647336 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/15d1835a-24a8-4c8b-9c73-4afba0b63366-data-volume\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:15.663957 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:15.663934 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98cdb\" (UniqueName: \"kubernetes.io/projected/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-api-access-98cdb\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:16.151246 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:16.151219 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:16.151581 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:16.151298 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:16.151581 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:16.151326 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 19:16:16.151581 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:16.151377 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm podName:15d1835a-24a8-4c8b-9c73-4afba0b63366 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:17.151361637 +0000 UTC m=+123.055709407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqxts" (UID: "15d1835a-24a8-4c8b-9c73-4afba0b63366") : configmap "kube-rbac-proxy" not found Sep 18 19:16:16.151581 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:16.151429 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 19:16:16.151581 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:16.151498 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls podName:15d1835a-24a8-4c8b-9c73-4afba0b63366 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:17.151465351 +0000 UTC m=+123.055813092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqxts" (UID: "15d1835a-24a8-4c8b-9c73-4afba0b63366") : secret "insights-runtime-extractor-tls" not found Sep 18 19:16:16.179720 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:16.179694 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:16.361360 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:16.361326 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:16.480202 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:16.480113 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:16.876105 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:16.876023 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:17.023556 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:17.023524 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:17.161262 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:17.161230 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:17.161631 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:17.161298 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:17.161631 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:17.161386 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 19:16:17.161631 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:17.161449 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls podName:15d1835a-24a8-4c8b-9c73-4afba0b63366 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:19.16143385 +0000 UTC m=+125.065781585 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqxts" (UID: "15d1835a-24a8-4c8b-9c73-4afba0b63366") : secret "insights-runtime-extractor-tls" not found Sep 18 19:16:17.161631 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:17.161389 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 19:16:17.161631 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:17.161512 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm podName:15d1835a-24a8-4c8b-9c73-4afba0b63366 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:19.161499234 +0000 UTC m=+125.065846990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqxts" (UID: "15d1835a-24a8-4c8b-9c73-4afba0b63366") : configmap "kube-rbac-proxy" not found Sep 18 19:16:17.362539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:17.362501 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:17.362539 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:17.362537 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:17.362792 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:17.362681 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 19:16:17.362792 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:17.362710 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:25.362688633 +0000 UTC m=+131.267036384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : configmap references non-existent config key: service-ca.crt Sep 18 19:16:17.362792 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:17.362739 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:25.362728659 +0000 UTC m=+131.267076394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : secret "router-metrics-certs-default" not found Sep 18 19:16:17.467411 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:17.467301 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v99gh_f6f93841-9b28-41ca-aab2-4caefd768842/dns-node-resolver/0.log" Sep 18 19:16:17.467572 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:17.467530 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:17.590744 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:17.590712 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:17.665653 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:17.665599 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:17.665811 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:17.665740 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 19:16:17.665811 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:17.665755 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598d87595f-rn9x8: secret "image-registry-tls" not found Sep 18 19:16:17.665811 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:17.665802 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls podName:8a7681ba-500b-466c-9094-0d160a51d4fa nodeName:}" failed. No retries permitted until 2026-09-18 19:16:25.665789829 +0000 UTC m=+131.570137569 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls") pod "image-registry-598d87595f-rn9x8" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa") : secret "image-registry-tls" not found Sep 18 19:16:18.248891 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.248867 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9fpjr_bc2c0c9b-3578-46cf-a91e-b3eeef557f64/node-ca/0.log" Sep 18 19:16:18.702485 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.702449 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:18.721754 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.721726 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-6j4d5"] Sep 18 19:16:18.723532 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.723518 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" Sep 18 19:16:18.726852 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.726835 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 18 19:16:18.727669 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.727632 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 18 19:16:18.727832 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.727817 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-kmb4g\"" Sep 18 19:16:18.727904 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.727885 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 18 19:16:18.727966 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.727902 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 18 19:16:18.857846 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.857817 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n2xhf_879e230b-7166-438e-b7d4-e88e9116a54b/serve-healthcheck-canary/0.log" Sep 18 19:16:18.858025 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.857982 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:18.875001 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.874983 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/562c7149-51ec-4272-be2d-a70b07c7b28e-signing-cabundle\") pod \"service-ca-55f6fc7b86-6j4d5\" (UID: \"562c7149-51ec-4272-be2d-a70b07c7b28e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" Sep 18 19:16:18.875063 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.875010 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/562c7149-51ec-4272-be2d-a70b07c7b28e-signing-key\") pod \"service-ca-55f6fc7b86-6j4d5\" (UID: \"562c7149-51ec-4272-be2d-a70b07c7b28e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" Sep 18 19:16:18.875063 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.875052 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tw4j\" (UniqueName: \"kubernetes.io/projected/562c7149-51ec-4272-be2d-a70b07c7b28e-kube-api-access-5tw4j\") pod \"service-ca-55f6fc7b86-6j4d5\" (UID: \"562c7149-51ec-4272-be2d-a70b07c7b28e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" Sep 18 19:16:18.905496 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.905467 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-6j4d5"] Sep 18 19:16:18.976073 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.975989 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/562c7149-51ec-4272-be2d-a70b07c7b28e-signing-cabundle\") pod \"service-ca-55f6fc7b86-6j4d5\" (UID: \"562c7149-51ec-4272-be2d-a70b07c7b28e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" Sep 18 19:16:18.976073 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.976024 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/562c7149-51ec-4272-be2d-a70b07c7b28e-signing-key\") pod \"service-ca-55f6fc7b86-6j4d5\" (UID: \"562c7149-51ec-4272-be2d-a70b07c7b28e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" Sep 18 19:16:18.976248 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.976082 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tw4j\" (UniqueName: \"kubernetes.io/projected/562c7149-51ec-4272-be2d-a70b07c7b28e-kube-api-access-5tw4j\") pod \"service-ca-55f6fc7b86-6j4d5\" (UID: \"562c7149-51ec-4272-be2d-a70b07c7b28e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" Sep 18 19:16:18.976748 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.976728 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/562c7149-51ec-4272-be2d-a70b07c7b28e-signing-cabundle\") pod \"service-ca-55f6fc7b86-6j4d5\" (UID: \"562c7149-51ec-4272-be2d-a70b07c7b28e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" Sep 18 19:16:18.978452 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.978429 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/562c7149-51ec-4272-be2d-a70b07c7b28e-signing-key\") pod \"service-ca-55f6fc7b86-6j4d5\" (UID: \"562c7149-51ec-4272-be2d-a70b07c7b28e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" Sep 18 19:16:18.991520 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:18.991498 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tw4j\" (UniqueName: \"kubernetes.io/projected/562c7149-51ec-4272-be2d-a70b07c7b28e-kube-api-access-5tw4j\") pod \"service-ca-55f6fc7b86-6j4d5\" (UID: \"562c7149-51ec-4272-be2d-a70b07c7b28e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" Sep 18 19:16:19.032536 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:19.032509 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" Sep 18 19:16:19.178294 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:19.178246 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:19.178474 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:19.178320 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:19.178474 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:19.178363 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 19:16:19.178474 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:19.178432 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm podName:15d1835a-24a8-4c8b-9c73-4afba0b63366 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:23.178416349 +0000 UTC m=+129.082764112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqxts" (UID: "15d1835a-24a8-4c8b-9c73-4afba0b63366") : configmap "kube-rbac-proxy" not found Sep 18 19:16:19.178626 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:19.178476 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 19:16:19.178626 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:19.178541 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls podName:15d1835a-24a8-4c8b-9c73-4afba0b63366 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:23.178524489 +0000 UTC m=+129.082872241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqxts" (UID: "15d1835a-24a8-4c8b-9c73-4afba0b63366") : secret "insights-runtime-extractor-tls" not found Sep 18 19:16:19.344886 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:19.344808 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-6j4d5"] Sep 18 19:16:19.348395 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:19.348370 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod562c7149_51ec_4272_be2d_a70b07c7b28e.slice/crio-5af2471ad03975eeb249389bd0378c6c787d11855930b723e065a9840fdd6278 WatchSource:0}: Error finding container 5af2471ad03975eeb249389bd0378c6c787d11855930b723e065a9840fdd6278: Status 404 returned error can't find the container with id 5af2471ad03975eeb249389bd0378c6c787d11855930b723e065a9840fdd6278 Sep 18 19:16:19.771656 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:19.771608 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" event={"ID":"562c7149-51ec-4272-be2d-a70b07c7b28e","Type":"ContainerStarted","Data":"5af2471ad03975eeb249389bd0378c6c787d11855930b723e065a9840fdd6278"} Sep 18 19:16:20.034201 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:20.034123 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:21.780124 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:21.780090 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" event={"ID":"562c7149-51ec-4272-be2d-a70b07c7b28e","Type":"ContainerStarted","Data":"5f4604587a3c3549d2b53c996fb6ea9c8dee5d8eb50e03f4b2ac47b4f1ad745b"} Sep 18 19:16:22.030776 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:22.030658 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:22.314276 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:22.314189 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:22.411375 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:22.411345 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:22.517350 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:22.517297 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:22.621751 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:22.621673 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:22.754547 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:22.754513 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:23.214689 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:23.214654 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:23.215048 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:23.214707 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:23.215048 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:23.214805 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 19:16:23.215048 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:23.214852 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls podName:15d1835a-24a8-4c8b-9c73-4afba0b63366 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:31.214840092 +0000 UTC m=+137.119187827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqxts" (UID: "15d1835a-24a8-4c8b-9c73-4afba0b63366") : secret "insights-runtime-extractor-tls" not found Sep 18 19:16:23.215048 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:23.214883 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 19:16:23.215048 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:23.214969 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm podName:15d1835a-24a8-4c8b-9c73-4afba0b63366 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:31.214954029 +0000 UTC m=+137.119301790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqxts" (UID: "15d1835a-24a8-4c8b-9c73-4afba0b63366") : configmap "kube-rbac-proxy" not found Sep 18 19:16:23.520883 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:23.520794 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:23.923070 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:23.923041 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:24.347394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:24.347309 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:24.722037 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:24.721995 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:25.113311 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:25.113233 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:25.305839 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:25.305801 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:25.432383 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:25.432333 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:25.432853 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:25.432439 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:25.432853 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:25.432502 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:41.432485387 +0000 UTC m=+147.336833125 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : configmap references non-existent config key: service-ca.crt Sep 18 19:16:25.432853 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:25.432561 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 19:16:25.432853 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:25.432621 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs podName:2cee06b4-79a2-47d4-9443-e1d1353ccc82 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:41.432604074 +0000 UTC m=+147.336951813 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs") pod "router-default-54788fddc8-7b577" (UID: "2cee06b4-79a2-47d4-9443-e1d1353ccc82") : secret "router-metrics-certs-default" not found Sep 18 19:16:25.709491 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:25.709391 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:25.734880 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:25.734848 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls\") pod \"image-registry-598d87595f-rn9x8\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:25.735013 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:25.734998 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 19:16:25.735060 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:25.735015 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598d87595f-rn9x8: secret "image-registry-tls" not found Sep 18 19:16:25.735145 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:25.735069 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls podName:8a7681ba-500b-466c-9094-0d160a51d4fa nodeName:}" failed. No retries permitted until 2026-09-18 19:16:41.735053477 +0000 UTC m=+147.639401226 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls") pod "image-registry-598d87595f-rn9x8" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa") : secret "image-registry-tls" not found Sep 18 19:16:25.907049 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:25.907012 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:25.937154 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:25.937125 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:16:25.937275 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:25.937261 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 18 19:16:25.937324 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:25.937317 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs podName:0899c67e-8d96-4fa0-b310-b6641c91127b nodeName:}" failed. No retries permitted until 2026-09-18 19:18:27.937302595 +0000 UTC m=+253.841650330 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs") pod "network-metrics-daemon-dvkb2" (UID: "0899c67e-8d96-4fa0-b310-b6641c91127b") : secret "metrics-daemon-secret" not found Sep 18 19:16:26.103734 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:26.103634 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:26.528563 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:26.528526 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:26.709047 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:26.709001 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:26.871025 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:26.870942 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:26.950608 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:26.950581 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:27.111716 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:27.111684 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:27.909865 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:27.909827 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:28.443211 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:28.443175 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:28.541324 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:28.541292 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:28.745627 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:28.745522 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:29.146978 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:29.146949 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:29.339951 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:29.339918 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:29.724561 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:29.724533 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:29.794919 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:29.794885 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:29.919997 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:29.919967 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:30.522989 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:30.522957 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:30.733650 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:30.733606 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:31.105060 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:31.105028 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:31.284455 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:31.284417 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:31.284659 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:31.284476 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:31.284659 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:31.284610 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 19:16:31.284782 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:31.284703 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls podName:15d1835a-24a8-4c8b-9c73-4afba0b63366 nodeName:}" failed. No retries permitted until 2026-09-18 19:16:47.284682152 +0000 UTC m=+153.189029909 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqxts" (UID: "15d1835a-24a8-4c8b-9c73-4afba0b63366") : secret "insights-runtime-extractor-tls" not found Sep 18 19:16:31.285125 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:31.285099 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15d1835a-24a8-4c8b-9c73-4afba0b63366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:31.319333 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:31.319300 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:31.708968 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:31.708938 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:32.216704 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:32.216675 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:32.308157 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:32.308111 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:32.449559 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:32.449525 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:32.549609 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:32.549529 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:32.651169 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:32.651135 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:32.750709 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:32.750675 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:32.851891 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:32.851813 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:32.956235 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:32.956193 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:33.251284 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:33.251254 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:34.034827 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:34.034768 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:34.248341 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:34.248312 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:34.389498 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:34.389470 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:35.655707 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:35.655671 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:35.734426 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:35.734394 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:37.204574 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.204518 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-6j4d5" podStartSLOduration=17.638158441 podStartE2EDuration="19.204501155s" totalImagesPullingTime="1.566342714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:21.809266425 +0000 UTC m=+127.713614182" watchObservedRunningTime="2026-09-18 19:16:37.204501155 +0000 UTC m=+143.108848912" Sep 18 19:16:37.205207 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.205187 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-598d87595f-rn9x8"] Sep 18 19:16:37.205418 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:37.205401 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-598d87595f-rn9x8" podUID="8a7681ba-500b-466c-9094-0d160a51d4fa" Sep 18 19:16:37.348966 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.348905 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:37.459286 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.459205 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:37.827760 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.827680 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:37.833840 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.833822 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:37.935048 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.935010 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-image-registry-private-configuration\") pod \"8a7681ba-500b-466c-9094-0d160a51d4fa\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " Sep 18 19:16:37.935048 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.935047 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tld5f\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-kube-api-access-tld5f\") pod \"8a7681ba-500b-466c-9094-0d160a51d4fa\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " Sep 18 19:16:37.935288 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.935074 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-certificates\") pod \"8a7681ba-500b-466c-9094-0d160a51d4fa\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " Sep 18 19:16:37.935288 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.935117 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-bound-sa-token\") pod \"8a7681ba-500b-466c-9094-0d160a51d4fa\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " Sep 18 19:16:37.935288 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.935132 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-trusted-ca\") pod \"8a7681ba-500b-466c-9094-0d160a51d4fa\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " Sep 18 19:16:37.935288 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.935151 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8a7681ba-500b-466c-9094-0d160a51d4fa-ca-trust-extracted\") pod \"8a7681ba-500b-466c-9094-0d160a51d4fa\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " Sep 18 19:16:37.935288 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.935178 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-installation-pull-secrets\") pod \"8a7681ba-500b-466c-9094-0d160a51d4fa\" (UID: \"8a7681ba-500b-466c-9094-0d160a51d4fa\") " Sep 18 19:16:37.936057 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.935783 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8a7681ba-500b-466c-9094-0d160a51d4fa" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:16:37.936057 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.936005 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7681ba-500b-466c-9094-0d160a51d4fa-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8a7681ba-500b-466c-9094-0d160a51d4fa" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 19:16:37.936254 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.936063 2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-certificates\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:16:37.936254 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.936008 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8a7681ba-500b-466c-9094-0d160a51d4fa" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:16:37.940505 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.938231 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8a7681ba-500b-466c-9094-0d160a51d4fa" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:16:37.940505 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.938344 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-kube-api-access-tld5f" (OuterVolumeSpecName: "kube-api-access-tld5f") pod "8a7681ba-500b-466c-9094-0d160a51d4fa" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa"). InnerVolumeSpecName "kube-api-access-tld5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 19:16:37.940505 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.938357 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8a7681ba-500b-466c-9094-0d160a51d4fa" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:16:37.940909 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:37.940882 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8a7681ba-500b-466c-9094-0d160a51d4fa" (UID: "8a7681ba-500b-466c-9094-0d160a51d4fa"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 19:16:38.036860 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:38.036827 2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-bound-sa-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:16:38.036860 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:38.036852 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8a7681ba-500b-466c-9094-0d160a51d4fa-trusted-ca\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:16:38.036860 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:38.036861 2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8a7681ba-500b-466c-9094-0d160a51d4fa-ca-trust-extracted\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:16:38.037079 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:38.036872 2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-installation-pull-secrets\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:16:38.037079 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:38.036883 2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8a7681ba-500b-466c-9094-0d160a51d4fa-image-registry-private-configuration\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:16:38.037079 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:38.036892 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tld5f\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-kube-api-access-tld5f\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:16:38.830912 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:38.830881 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-598d87595f-rn9x8" Sep 18 19:16:38.896011 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:38.895981 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-598d87595f-rn9x8"] Sep 18 19:16:38.901498 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:38.901476 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-598d87595f-rn9x8"] Sep 18 19:16:38.942398 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:38.942373 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8a7681ba-500b-466c-9094-0d160a51d4fa-registry-tls\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:16:40.202852 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:40.202798 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:40.275120 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:40.275092 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a7681ba-500b-466c-9094-0d160a51d4fa" path="/var/lib/kubelet/pods/8a7681ba-500b-466c-9094-0d160a51d4fa/volumes" Sep 18 19:16:40.287493 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:40.287467 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:40.379097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:40.379073 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:41.462779 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:41.462744 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:41.463270 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:41.462804 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:41.463500 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:41.463478 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cee06b4-79a2-47d4-9443-e1d1353ccc82-service-ca-bundle\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:41.465059 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:41.465041 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cee06b4-79a2-47d4-9443-e1d1353ccc82-metrics-certs\") pod \"router-default-54788fddc8-7b577\" (UID: \"2cee06b4-79a2-47d4-9443-e1d1353ccc82\") " pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:41.695285 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:41.695252 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-27vxx\"" Sep 18 19:16:41.703306 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:41.703274 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:41.980483 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:41.980401 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54788fddc8-7b577"] Sep 18 19:16:41.983782 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:41.983759 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cee06b4_79a2_47d4_9443_e1d1353ccc82.slice/crio-c517d01839f00117a9ee78475bc18ea39b757b22520fc4e86afc07c8582e5483 WatchSource:0}: Error finding container c517d01839f00117a9ee78475bc18ea39b757b22520fc4e86afc07c8582e5483: Status 404 returned error can't find the container with id c517d01839f00117a9ee78475bc18ea39b757b22520fc4e86afc07c8582e5483 Sep 18 19:16:42.843470 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:42.843434 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54788fddc8-7b577" event={"ID":"2cee06b4-79a2-47d4-9443-e1d1353ccc82","Type":"ContainerStarted","Data":"8a71453cb52cf18b299bf774ca567a5f811706cbc1714206585e47b806557b8e"} Sep 18 19:16:42.843884 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:42.843483 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54788fddc8-7b577" event={"ID":"2cee06b4-79a2-47d4-9443-e1d1353ccc82","Type":"ContainerStarted","Data":"c517d01839f00117a9ee78475bc18ea39b757b22520fc4e86afc07c8582e5483"} Sep 18 19:16:42.880474 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:42.880431 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-54788fddc8-7b577" podStartSLOduration=33.880415766 podStartE2EDuration="33.880415766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:42.879818071 +0000 UTC m=+148.784165846" watchObservedRunningTime="2026-09-18 19:16:42.880415766 +0000 UTC m=+148.784763529" Sep 18 19:16:43.530385 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.530331 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:43.704038 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.703999 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:43.706883 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.706860 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:43.769422 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.769390 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:43.810079 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.809999 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt"] Sep 18 19:16:43.813021 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.813006 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt" Sep 18 19:16:43.822036 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.822018 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 18 19:16:43.822163 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.822086 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-d6rhz\"" Sep 18 19:16:43.844160 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.844134 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt"] Sep 18 19:16:43.846622 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.846587 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:43.847860 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.847835 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-54788fddc8-7b577" Sep 18 19:16:43.878924 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.878895 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0907315e-5c19-424a-b15b-6bfce3a15da7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-vkktt\" (UID: \"0907315e-5c19-424a-b15b-6bfce3a15da7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt" Sep 18 19:16:43.911832 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.911806 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:43.979940 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.979910 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0907315e-5c19-424a-b15b-6bfce3a15da7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-vkktt\" (UID: \"0907315e-5c19-424a-b15b-6bfce3a15da7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt" Sep 18 19:16:43.982328 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:43.982301 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0907315e-5c19-424a-b15b-6bfce3a15da7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-vkktt\" (UID: \"0907315e-5c19-424a-b15b-6bfce3a15da7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt" Sep 18 19:16:44.121289 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:44.121197 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt" Sep 18 19:16:44.406130 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:44.406100 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt"] Sep 18 19:16:44.409059 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:44.409032 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0907315e_5c19_424a_b15b_6bfce3a15da7.slice/crio-78f117ce24cd417e6607b7f03591ecf9c2ae01fdfcd75d794f8f051e404557b6 WatchSource:0}: Error finding container 78f117ce24cd417e6607b7f03591ecf9c2ae01fdfcd75d794f8f051e404557b6: Status 404 returned error can't find the container with id 78f117ce24cd417e6607b7f03591ecf9c2ae01fdfcd75d794f8f051e404557b6 Sep 18 19:16:44.698342 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:44.698260 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:44.816102 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:44.816061 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:44.850559 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:44.850528 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt" event={"ID":"0907315e-5c19-424a-b15b-6bfce3a15da7","Type":"ContainerStarted","Data":"78f117ce24cd417e6607b7f03591ecf9c2ae01fdfcd75d794f8f051e404557b6"} Sep 18 19:16:45.854695 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:45.854599 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt" event={"ID":"0907315e-5c19-424a-b15b-6bfce3a15da7","Type":"ContainerStarted","Data":"5b7fdd8f4e820f2337fdf3642b3d654796d63d9f30ad73eb1b040ee6271078ec"} Sep 18 19:16:45.901727 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:45.901675 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt" podStartSLOduration=1.887417125 podStartE2EDuration="2.901660133s" totalImagesPullingTime="1.014243008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:45.900619686 +0000 UTC m=+151.804967442" watchObservedRunningTime="2026-09-18 19:16:45.901660133 +0000 UTC m=+151.806007887" Sep 18 19:16:46.857310 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:46.857280 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt" Sep 18 19:16:46.861799 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:46.861770 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-vkktt" Sep 18 19:16:47.304008 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.303970 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:47.306333 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.306307 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15d1835a-24a8-4c8b-9c73-4afba0b63366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqxts\" (UID: \"15d1835a-24a8-4c8b-9c73-4afba0b63366\") " pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:47.342936 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.342898 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:47.452417 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.452380 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bf8d7d659-wz2zs"] Sep 18 19:16:47.464517 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.464491 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:47.496509 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.496482 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.503272 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.503251 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bf8d7d659-wz2zs"] Sep 18 19:16:47.511430 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.511407 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 18 19:16:47.511560 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.511457 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 18 19:16:47.512104 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.512081 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 18 19:16:47.512534 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.512518 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jjztf\"" Sep 18 19:16:47.512799 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.512786 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 18 19:16:47.512947 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.512930 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 18 19:16:47.513154 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.513141 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 18 19:16:47.514702 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.514690 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 18 19:16:47.550086 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.550054 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-npv6k\"" Sep 18 19:16:47.556769 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.556720 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wqxts" Sep 18 19:16:47.605673 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.605627 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-oauth-config\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.605819 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.605706 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-oauth-serving-cert\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.605819 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.605746 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-serving-cert\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.605819 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.605785 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn6p6\" (UniqueName: \"kubernetes.io/projected/ceaa43c6-d36b-4435-a11c-5c51981d9140-kube-api-access-tn6p6\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.605941 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.605830 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-config\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.605941 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.605861 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-service-ca\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.707052 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.707018 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-oauth-config\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.707052 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.707053 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-oauth-serving-cert\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.707282 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.707080 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-serving-cert\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.707282 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.707124 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn6p6\" (UniqueName: \"kubernetes.io/projected/ceaa43c6-d36b-4435-a11c-5c51981d9140-kube-api-access-tn6p6\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.707282 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.707154 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-config\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.707282 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.707191 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-service-ca\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.707864 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.707836 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-oauth-serving-cert\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.707985 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.707840 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-service-ca\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.707985 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.707926 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-config\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.709494 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.709464 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-serving-cert\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.709615 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.709501 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-oauth-config\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.724537 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.724505 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn6p6\" (UniqueName: \"kubernetes.io/projected/ceaa43c6-d36b-4435-a11c-5c51981d9140-kube-api-access-tn6p6\") pod \"console-7bf8d7d659-wz2zs\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.805303 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.805262 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:47.874459 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.874428 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wqxts"] Sep 18 19:16:47.879118 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:47.879087 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d1835a_24a8_4c8b_9c73_4afba0b63366.slice/crio-f4fc7779c46f93e03ae56a728746ea34be3b40b2dec917afb8a598788dac7ce3 WatchSource:0}: Error finding container f4fc7779c46f93e03ae56a728746ea34be3b40b2dec917afb8a598788dac7ce3: Status 404 returned error can't find the container with id f4fc7779c46f93e03ae56a728746ea34be3b40b2dec917afb8a598788dac7ce3 Sep 18 19:16:47.893437 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:47.893408 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:48.034422 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.034393 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:48.089680 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.089587 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb"] Sep 18 19:16:48.115071 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.115038 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.118020 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.117980 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 18 19:16:48.118153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.118076 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 18 19:16:48.118258 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.118243 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-h2bdw\"" Sep 18 19:16:48.118414 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.118401 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 18 19:16:48.118519 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.118502 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 18 19:16:48.122188 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.122120 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb"] Sep 18 19:16:48.137393 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.137364 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 18 19:16:48.149598 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.149563 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bf8d7d659-wz2zs"] Sep 18 19:16:48.150911 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:48.150887 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceaa43c6_d36b_4435_a11c_5c51981d9140.slice/crio-ec7d0a3f24b0831db8fa730575ddb5ba8c833697913142a9778af6c60853a1ac WatchSource:0}: Error finding container ec7d0a3f24b0831db8fa730575ddb5ba8c833697913142a9778af6c60853a1ac: Status 404 returned error can't find the container with id ec7d0a3f24b0831db8fa730575ddb5ba8c833697913142a9778af6c60853a1ac Sep 18 19:16:48.202493 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.202463 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:48.209595 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.209571 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.209694 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.209608 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.209734 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.209716 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4dzc\" (UniqueName: \"kubernetes.io/projected/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-kube-api-access-n4dzc\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.209766 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.209747 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.310211 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.310175 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.310211 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.310216 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.310431 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.310268 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4dzc\" (UniqueName: \"kubernetes.io/projected/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-kube-api-access-n4dzc\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.310431 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.310373 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.311027 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.311005 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.312685 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.312666 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.312771 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.312673 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.323967 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.323945 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4dzc\" (UniqueName: \"kubernetes.io/projected/331a3f0f-55b3-4e1d-83ec-e6fea064ffd5-kube-api-access-n4dzc\") pod \"prometheus-operator-7bd5d56d6-dv7rb\" (UID: \"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.425158 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.425122 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" Sep 18 19:16:48.735425 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.735338 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb"] Sep 18 19:16:48.739228 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:48.739191 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod331a3f0f_55b3_4e1d_83ec_e6fea064ffd5.slice/crio-f33d83a079588c16d44294a002909288c021efa3b7994fe2b743be50386670ec WatchSource:0}: Error finding container f33d83a079588c16d44294a002909288c021efa3b7994fe2b743be50386670ec: Status 404 returned error can't find the container with id f33d83a079588c16d44294a002909288c021efa3b7994fe2b743be50386670ec Sep 18 19:16:48.867968 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.867919 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqxts" event={"ID":"15d1835a-24a8-4c8b-9c73-4afba0b63366","Type":"ContainerStarted","Data":"5e9f4fe56c042b54625495c3f986e1454681369b6ec95f4fda13c595cd5c770a"} Sep 18 19:16:48.868149 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.867998 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqxts" event={"ID":"15d1835a-24a8-4c8b-9c73-4afba0b63366","Type":"ContainerStarted","Data":"90429354470f8485f6744e2c94aa7441a7596cd518dd8ed2a84688ee0fd9711f"} Sep 18 19:16:48.868149 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.868031 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqxts" event={"ID":"15d1835a-24a8-4c8b-9c73-4afba0b63366","Type":"ContainerStarted","Data":"f4fc7779c46f93e03ae56a728746ea34be3b40b2dec917afb8a598788dac7ce3"} Sep 18 19:16:48.870252 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.870222 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf8d7d659-wz2zs" event={"ID":"ceaa43c6-d36b-4435-a11c-5c51981d9140","Type":"ContainerStarted","Data":"ec7d0a3f24b0831db8fa730575ddb5ba8c833697913142a9778af6c60853a1ac"} Sep 18 19:16:48.871833 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:48.871806 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" event={"ID":"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5","Type":"ContainerStarted","Data":"f33d83a079588c16d44294a002909288c021efa3b7994fe2b743be50386670ec"} Sep 18 19:16:49.539304 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:49.539208 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:49.633235 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:49.633195 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:51.510182 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:51.510135 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:51.619317 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:51.619209 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:52.579190 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:52.579136 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rbmm8" podUID="9996e7d0-59d0-4457-8967-9a4c7eb67967" Sep 18 19:16:52.887047 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:52.887013 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rbmm8" Sep 18 19:16:53.274589 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:53.274503 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:53.282520 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:16:53.282500 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dvkb2" podUID="0899c67e-8d96-4fa0-b310-b6641c91127b" Sep 18 19:16:53.363105 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:53.363075 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:53.893406 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:53.893374 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqxts" event={"ID":"15d1835a-24a8-4c8b-9c73-4afba0b63366","Type":"ContainerStarted","Data":"ab15e13891dba7c9b78161434d9c253be39b350d063fc11f5f847fb9a4a75fec"} Sep 18 19:16:53.894966 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:53.894939 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf8d7d659-wz2zs" event={"ID":"ceaa43c6-d36b-4435-a11c-5c51981d9140","Type":"ContainerStarted","Data":"642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863"} Sep 18 19:16:53.896567 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:53.896545 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" event={"ID":"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5","Type":"ContainerStarted","Data":"3113a88e4dddd11e89a7aa16f7bb735ca152c1d268ddb60683d4133ca91c64e6"} Sep 18 19:16:53.896677 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:53.896584 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" event={"ID":"331a3f0f-55b3-4e1d-83ec-e6fea064ffd5","Type":"ContainerStarted","Data":"d30222bfebfec95e9408fd9e97dd7b866a34d12dac84aa449c30ab326ebc32c8"} Sep 18 19:16:53.926060 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:53.926015 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wqxts" podStartSLOduration=34.00528877 podStartE2EDuration="38.925999381s" totalImagesPullingTime="4.920710611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:16:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:53.924009042 +0000 UTC m=+159.828356814" watchObservedRunningTime="2026-09-18 19:16:53.925999381 +0000 UTC m=+159.830347137" Sep 18 19:16:53.960989 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:53.960947 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dv7rb" podStartSLOduration=1.734407038 podStartE2EDuration="5.960933131s" totalImagesPullingTime="4.226526093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:53.960444829 +0000 UTC m=+159.864792606" watchObservedRunningTime="2026-09-18 19:16:53.960933131 +0000 UTC m=+159.865280887" Sep 18 19:16:55.178178 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.178140 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:55.459357 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.459263 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:55.553404 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.553337 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bf8d7d659-wz2zs" podStartSLOduration=3.678499997 podStartE2EDuration="8.55331551s" totalImagesPullingTime="4.874815513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:53.988244287 +0000 UTC m=+159.892592044" watchObservedRunningTime="2026-09-18 19:16:55.55331551 +0000 UTC m=+161.457663269" Sep 18 19:16:55.554506 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.554480 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd"] Sep 18 19:16:55.560576 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.560558 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.563158 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.563138 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tddvb\"" Sep 18 19:16:55.563602 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.563587 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 18 19:16:55.572369 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.572343 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:55.575420 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.575403 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 18 19:16:55.582632 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.582607 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd"] Sep 18 19:16:55.640861 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.640831 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fj5wk"] Sep 18 19:16:55.644722 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.644698 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.650197 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.650178 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 18 19:16:55.650602 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.650579 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 18 19:16:55.650828 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.650804 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 18 19:16:55.650926 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.650833 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nzvgt\"" Sep 18 19:16:55.672838 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.672806 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-wtmp\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.672949 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.672842 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmbm7\" (UniqueName: \"kubernetes.io/projected/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-kube-api-access-zmbm7\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.672949 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.672867 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67873421-8959-4e5d-a3b0-e2b650ff1292-root\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.673018 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.672960 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-tls\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.673018 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.673004 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.673125 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.673103 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.673162 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.673143 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67873421-8959-4e5d-a3b0-e2b650ff1292-metrics-client-ca\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.673194 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.673173 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rmvp\" (UniqueName: \"kubernetes.io/projected/67873421-8959-4e5d-a3b0-e2b650ff1292-kube-api-access-6rmvp\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.673226 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.673199 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.673259 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.673236 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67873421-8959-4e5d-a3b0-e2b650ff1292-sys\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.673293 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.673277 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-textfile\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.673329 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.673314 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-accelerators-collector-config\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.673384 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.673368 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.685145 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.685098 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:55.773876 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.773804 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.773876 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.773842 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-wtmp\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.773876 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.773866 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmbm7\" (UniqueName: \"kubernetes.io/projected/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-kube-api-access-zmbm7\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.774153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.773892 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67873421-8959-4e5d-a3b0-e2b650ff1292-root\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.773918 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-tls\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.773952 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.774153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.773989 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-wtmp\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.773998 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774030 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67873421-8959-4e5d-a3b0-e2b650ff1292-metrics-client-ca\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774058 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rmvp\" (UniqueName: \"kubernetes.io/projected/67873421-8959-4e5d-a3b0-e2b650ff1292-kube-api-access-6rmvp\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774082 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.774153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774032 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67873421-8959-4e5d-a3b0-e2b650ff1292-root\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774153 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774124 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67873421-8959-4e5d-a3b0-e2b650ff1292-sys\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774675 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774189 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67873421-8959-4e5d-a3b0-e2b650ff1292-sys\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774675 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774312 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-textfile\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774675 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774371 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-accelerators-collector-config\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774833 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774780 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67873421-8959-4e5d-a3b0-e2b650ff1292-metrics-client-ca\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774892 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774863 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.774892 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774869 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-textfile\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.774997 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.774928 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-accelerators-collector-config\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.776464 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.776433 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-tls\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.776587 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.776434 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67873421-8959-4e5d-a3b0-e2b650ff1292-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.776587 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.776470 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.776789 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.776768 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.788567 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.788542 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rmvp\" (UniqueName: \"kubernetes.io/projected/67873421-8959-4e5d-a3b0-e2b650ff1292-kube-api-access-6rmvp\") pod \"node-exporter-fj5wk\" (UID: \"67873421-8959-4e5d-a3b0-e2b650ff1292\") " pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.791449 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.791428 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:55.791538 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.791451 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmbm7\" (UniqueName: \"kubernetes.io/projected/976f4b92-c892-4be8-8cd8-db5d1f03bcc2-kube-api-access-zmbm7\") pod \"openshift-state-metrics-86c5bc79d6-754fd\" (UID: \"976f4b92-c892-4be8-8cd8-db5d1f03bcc2\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.869601 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.869568 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" Sep 18 19:16:55.902920 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.902876 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:55.955000 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:55.954971 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fj5wk" Sep 18 19:16:55.963278 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:55.963250 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67873421_8959_4e5d_a3b0_e2b650ff1292.slice/crio-ff50c93688374eb31cca52a4d94060ad92c55374b95cc38ce22c94a8a9e3c887 WatchSource:0}: Error finding container ff50c93688374eb31cca52a4d94060ad92c55374b95cc38ce22c94a8a9e3c887: Status 404 returned error can't find the container with id ff50c93688374eb31cca52a4d94060ad92c55374b95cc38ce22c94a8a9e3c887 Sep 18 19:16:56.048519 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:56.048432 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:56.146736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:56.146703 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd"] Sep 18 19:16:56.149477 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:56.149456 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod976f4b92_c892_4be8_8cd8_db5d1f03bcc2.slice/crio-16db2852073ca6f0d58a3cfae36c991661c43b03afffc578d76acdfe1c5f4907 WatchSource:0}: Error finding container 16db2852073ca6f0d58a3cfae36c991661c43b03afffc578d76acdfe1c5f4907: Status 404 returned error can't find the container with id 16db2852073ca6f0d58a3cfae36c991661c43b03afffc578d76acdfe1c5f4907 Sep 18 19:16:56.262805 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:56.262775 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:56.633815 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:56.633777 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:56.908868 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:56.908842 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj5wk" event={"ID":"67873421-8959-4e5d-a3b0-e2b650ff1292","Type":"ContainerStarted","Data":"ff50c93688374eb31cca52a4d94060ad92c55374b95cc38ce22c94a8a9e3c887"} Sep 18 19:16:56.910827 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:56.910808 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" event={"ID":"976f4b92-c892-4be8-8cd8-db5d1f03bcc2","Type":"ContainerStarted","Data":"74809ae2277ff890ae777425ecc51ee70af6c7a05a11a4c8dd1f92fb586885b0"} Sep 18 19:16:56.910908 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:56.910854 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" event={"ID":"976f4b92-c892-4be8-8cd8-db5d1f03bcc2","Type":"ContainerStarted","Data":"c966ce31f573c4eb2fc9e04d2bbf6f3841f7e0e765f60334f00fd282579d9f9f"} Sep 18 19:16:56.910908 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:56.910875 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" event={"ID":"976f4b92-c892-4be8-8cd8-db5d1f03bcc2","Type":"ContainerStarted","Data":"16db2852073ca6f0d58a3cfae36c991661c43b03afffc578d76acdfe1c5f4907"} Sep 18 19:16:57.161959 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.161847 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:57.387354 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.387319 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:16:57.389903 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.389876 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9996e7d0-59d0-4457-8967-9a4c7eb67967-metrics-tls\") pod \"dns-default-rbmm8\" (UID: \"9996e7d0-59d0-4457-8967-9a4c7eb67967\") " pod="openshift-dns/dns-default-rbmm8" Sep 18 19:16:57.412878 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.412808 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:57.692215 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.692116 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h7n8q\"" Sep 18 19:16:57.698103 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.698089 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rbmm8" Sep 18 19:16:57.806247 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.806219 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:57.806381 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.806304 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:16:57.807600 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.807577 2554 patch_prober.go:28] interesting pod/console-7bf8d7d659-wz2zs container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.15:8443/health\": dial tcp 10.133.0.15:8443: connect: connection refused" start-of-body= Sep 18 19:16:57.807709 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.807622 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7bf8d7d659-wz2zs" podUID="ceaa43c6-d36b-4435-a11c-5c51981d9140" containerName="console" probeResult="failure" output="Get \"https://10.133.0.15:8443/health\": dial tcp 10.133.0.15:8443: connect: connection refused" Sep 18 19:16:57.915936 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.915904 2554 generic.go:356] "Generic (PLEG): container finished" podID="67873421-8959-4e5d-a3b0-e2b650ff1292" containerID="13a0531f8dce46207fc3ae02150173720bf9fdd186397253886ed6328996dece" exitCode=0 Sep 18 19:16:57.916090 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.915995 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj5wk" event={"ID":"67873421-8959-4e5d-a3b0-e2b650ff1292","Type":"ContainerDied","Data":"13a0531f8dce46207fc3ae02150173720bf9fdd186397253886ed6328996dece"} Sep 18 19:16:57.918302 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.918278 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" event={"ID":"976f4b92-c892-4be8-8cd8-db5d1f03bcc2","Type":"ContainerStarted","Data":"ab1e50773a4eb3d52af0e3fc6e62b329985f79f9bc86e7e1b9a401eaac743e13"} Sep 18 19:16:57.994468 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:57.994418 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-754fd" podStartSLOduration=1.796458151 podStartE2EDuration="2.994400302s" totalImagesPullingTime="1.197942151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:57.993362232 +0000 UTC m=+163.897710020" watchObservedRunningTime="2026-09-18 19:16:57.994400302 +0000 UTC m=+163.898748058" Sep 18 19:16:58.002569 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.002548 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rbmm8"] Sep 18 19:16:58.005055 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:58.005025 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9996e7d0_59d0_4457_8967_9a4c7eb67967.slice/crio-5adfde90dc24b2d4a264b5c3db50a000c3b0a308691186f0fba17f4d79668306 WatchSource:0}: Error finding container 5adfde90dc24b2d4a264b5c3db50a000c3b0a308691186f0fba17f4d79668306: Status 404 returned error can't find the container with id 5adfde90dc24b2d4a264b5c3db50a000c3b0a308691186f0fba17f4d79668306 Sep 18 19:16:58.662970 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.662931 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-68665cdd46-v65g5"] Sep 18 19:16:58.666345 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.666325 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.672532 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.672507 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 18 19:16:58.672679 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.672524 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 18 19:16:58.672679 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.672586 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6fpse81foga50\"" Sep 18 19:16:58.672850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.672829 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 18 19:16:58.673220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.673202 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 18 19:16:58.681003 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.680982 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 18 19:16:58.681103 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.681042 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2f5rf\"" Sep 18 19:16:58.699568 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.699495 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-grpc-tls\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.699568 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.699544 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.699754 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.699602 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8gnf\" (UniqueName: \"kubernetes.io/projected/3ae18d24-a8fa-4605-bd57-0fb210b92845-kube-api-access-k8gnf\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.699754 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.699684 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae18d24-a8fa-4605-bd57-0fb210b92845-metrics-client-ca\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.699754 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.699743 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.699900 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.699787 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-tls\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.699900 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.699814 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.699977 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.699907 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.704554 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.704533 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68665cdd46-v65g5"] Sep 18 19:16:58.800310 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.800269 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae18d24-a8fa-4605-bd57-0fb210b92845-metrics-client-ca\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.800497 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.800322 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.800497 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.800361 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-tls\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.800497 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.800383 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.800497 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.800428 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.800497 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.800476 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-grpc-tls\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.800773 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.800504 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.800773 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.800539 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8gnf\" (UniqueName: \"kubernetes.io/projected/3ae18d24-a8fa-4605-bd57-0fb210b92845-kube-api-access-k8gnf\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.801148 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.801110 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae18d24-a8fa-4605-bd57-0fb210b92845-metrics-client-ca\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.803124 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.803094 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.803219 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.803146 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-tls\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.803272 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.803257 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.803553 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.803530 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.803612 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.803549 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.803715 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.803700 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ae18d24-a8fa-4605-bd57-0fb210b92845-secret-grpc-tls\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.813063 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.813037 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8gnf\" (UniqueName: \"kubernetes.io/projected/3ae18d24-a8fa-4605-bd57-0fb210b92845-kube-api-access-k8gnf\") pod \"thanos-querier-68665cdd46-v65g5\" (UID: \"3ae18d24-a8fa-4605-bd57-0fb210b92845\") " pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:58.913326 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.913248 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:58.923024 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.922992 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rbmm8" event={"ID":"9996e7d0-59d0-4457-8967-9a4c7eb67967","Type":"ContainerStarted","Data":"5adfde90dc24b2d4a264b5c3db50a000c3b0a308691186f0fba17f4d79668306"} Sep 18 19:16:58.925349 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.925316 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj5wk" event={"ID":"67873421-8959-4e5d-a3b0-e2b650ff1292","Type":"ContainerStarted","Data":"a88c9dba039dd621b28267d533b2cce2feb6f3000bcdd909e7ea353b605531ac"} Sep 18 19:16:58.925456 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.925378 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fj5wk" event={"ID":"67873421-8959-4e5d-a3b0-e2b650ff1292","Type":"ContainerStarted","Data":"a317d17dce2868cd857e9af644b5b10d6b7c137d9001cd901a5e7fc7acfe2e1f"} Sep 18 19:16:58.959837 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.959777 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fj5wk" podStartSLOduration=3.001443943 podStartE2EDuration="3.959756812s" totalImagesPullingTime="897.723253ms" totalInitContainerRuntime="60.589616ms" isStatefulPod=true podCreationTimestamp="2026-09-18 19:16:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:58.95833397 +0000 UTC m=+164.862681729" watchObservedRunningTime="2026-09-18 19:16:58.959756812 +0000 UTC m=+164.864104571" Sep 18 19:16:58.976768 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:58.976742 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:16:59.111307 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.111275 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:59.305227 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.305141 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:59.363381 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.363257 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68665cdd46-v65g5"] Sep 18 19:16:59.447416 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.447376 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:59.533134 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:16:59.533102 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae18d24_a8fa_4605_bd57_0fb210b92845.slice/crio-a19dd264a9bcab9345b95a79ff284907bdc81135488e7d0d3d512b5f43593495 WatchSource:0}: Error finding container a19dd264a9bcab9345b95a79ff284907bdc81135488e7d0d3d512b5f43593495: Status 404 returned error can't find the container with id a19dd264a9bcab9345b95a79ff284907bdc81135488e7d0d3d512b5f43593495 Sep 18 19:16:59.583330 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.583307 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:59.686359 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.686337 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:59.837325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.837242 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:16:59.931119 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.931085 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rbmm8" event={"ID":"9996e7d0-59d0-4457-8967-9a4c7eb67967","Type":"ContainerStarted","Data":"fa87b8f6f25519a5820c2b122f6a2d27bbe11902c35e5cf41db9b2b2627496ba"} Sep 18 19:16:59.931295 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.931140 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rbmm8" event={"ID":"9996e7d0-59d0-4457-8967-9a4c7eb67967","Type":"ContainerStarted","Data":"ad7b4c657c0254dac7128b2d2657fda297fce8656deb3ac21971ae4f409764cc"} Sep 18 19:16:59.931295 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.931170 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rbmm8" Sep 18 19:16:59.932379 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.932363 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" event={"ID":"3ae18d24-a8fa-4605-bd57-0fb210b92845","Type":"ContainerStarted","Data":"a19dd264a9bcab9345b95a79ff284907bdc81135488e7d0d3d512b5f43593495"} Sep 18 19:16:59.965761 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:16:59.965713 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rbmm8" podStartSLOduration=129.388603586 podStartE2EDuration="2m10.965695841s" totalImagesPullingTime="1.577092255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:14:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:59.965426658 +0000 UTC m=+165.869774435" watchObservedRunningTime="2026-09-18 19:16:59.965695841 +0000 UTC m=+165.870043598" Sep 18 19:17:00.318527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.318492 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:00.400088 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.400056 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg"] Sep 18 19:17:00.403320 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.403291 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" Sep 18 19:17:00.406876 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.406803 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-kzkqm\"" Sep 18 19:17:00.410113 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.410090 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 18 19:17:00.422858 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.422823 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg"] Sep 18 19:17:00.497754 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.497723 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:00.516915 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.516887 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6da8bc41-9fa5-4802-8f1f-08e96fed5f60-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-t7kxg\" (UID: \"6da8bc41-9fa5-4802-8f1f-08e96fed5f60\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" Sep 18 19:17:00.517066 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.516925 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jpz4\" (UniqueName: \"kubernetes.io/projected/6da8bc41-9fa5-4802-8f1f-08e96fed5f60-kube-api-access-7jpz4\") pod \"monitoring-plugin-5984459ff8-t7kxg\" (UID: \"6da8bc41-9fa5-4802-8f1f-08e96fed5f60\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" Sep 18 19:17:00.617431 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.617352 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6da8bc41-9fa5-4802-8f1f-08e96fed5f60-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-t7kxg\" (UID: \"6da8bc41-9fa5-4802-8f1f-08e96fed5f60\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" Sep 18 19:17:00.617431 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.617396 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jpz4\" (UniqueName: \"kubernetes.io/projected/6da8bc41-9fa5-4802-8f1f-08e96fed5f60-kube-api-access-7jpz4\") pod \"monitoring-plugin-5984459ff8-t7kxg\" (UID: \"6da8bc41-9fa5-4802-8f1f-08e96fed5f60\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" Sep 18 19:17:00.617656 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:17:00.617537 2554 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 18 19:17:00.617656 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:17:00.617614 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6da8bc41-9fa5-4802-8f1f-08e96fed5f60-monitoring-plugin-cert podName:6da8bc41-9fa5-4802-8f1f-08e96fed5f60 nodeName:}" failed. No retries permitted until 2026-09-18 19:17:01.117589942 +0000 UTC m=+167.021937681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/6da8bc41-9fa5-4802-8f1f-08e96fed5f60-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-t7kxg" (UID: "6da8bc41-9fa5-4802-8f1f-08e96fed5f60") : secret "monitoring-plugin-cert" not found Sep 18 19:17:00.633775 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.633745 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jpz4\" (UniqueName: \"kubernetes.io/projected/6da8bc41-9fa5-4802-8f1f-08e96fed5f60-kube-api-access-7jpz4\") pod \"monitoring-plugin-5984459ff8-t7kxg\" (UID: \"6da8bc41-9fa5-4802-8f1f-08e96fed5f60\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" Sep 18 19:17:00.803223 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:00.803174 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:01.122914 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:01.122879 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6da8bc41-9fa5-4802-8f1f-08e96fed5f60-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-t7kxg\" (UID: \"6da8bc41-9fa5-4802-8f1f-08e96fed5f60\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" Sep 18 19:17:01.125527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:01.125503 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6da8bc41-9fa5-4802-8f1f-08e96fed5f60-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-t7kxg\" (UID: \"6da8bc41-9fa5-4802-8f1f-08e96fed5f60\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" Sep 18 19:17:01.315175 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:01.315138 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" Sep 18 19:17:01.723997 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:01.723914 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg"] Sep 18 19:17:01.727239 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:17:01.727216 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da8bc41_9fa5_4802_8f1f_08e96fed5f60.slice/crio-a6656cc63ccc1306c66ffa5528ef39ea48eded47cece886d633eacc500273a76 WatchSource:0}: Error finding container a6656cc63ccc1306c66ffa5528ef39ea48eded47cece886d633eacc500273a76: Status 404 returned error can't find the container with id a6656cc63ccc1306c66ffa5528ef39ea48eded47cece886d633eacc500273a76 Sep 18 19:17:01.941934 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:01.941899 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" event={"ID":"6da8bc41-9fa5-4802-8f1f-08e96fed5f60","Type":"ContainerStarted","Data":"a6656cc63ccc1306c66ffa5528ef39ea48eded47cece886d633eacc500273a76"} Sep 18 19:17:01.944059 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:01.944037 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" event={"ID":"3ae18d24-a8fa-4605-bd57-0fb210b92845","Type":"ContainerStarted","Data":"c9d59e3a75026c21a196f4ee72f1956882f4eeb9824502663e145541ea89a9b4"} Sep 18 19:17:01.944181 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:01.944093 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" event={"ID":"3ae18d24-a8fa-4605-bd57-0fb210b92845","Type":"ContainerStarted","Data":"c6ab3ff373b351c4eec291e27d8b19b80f00c7cbd200e98d0279846a2a372c18"} Sep 18 19:17:01.944181 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:01.944125 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" event={"ID":"3ae18d24-a8fa-4605-bd57-0fb210b92845","Type":"ContainerStarted","Data":"de6ca06ebe9d7b249b86f964d8df3b41af28d6b3a8cb12fc140e9beacc4c5dec"} Sep 18 19:17:02.679832 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.679788 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:02.861662 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.861563 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 19:17:02.869221 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.869196 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.877365 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.877322 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 18 19:17:02.877365 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.877322 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 18 19:17:02.878134 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.877526 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 18 19:17:02.878134 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.877997 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 18 19:17:02.878398 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.878382 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 18 19:17:02.878525 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.878508 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 18 19:17:02.878692 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.878675 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 18 19:17:02.878901 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.878879 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 18 19:17:02.885128 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.885110 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 18 19:17:02.885235 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.885222 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 18 19:17:02.892264 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.892245 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b8bllk14hahl5\"" Sep 18 19:17:02.892367 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.892286 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 18 19:17:02.892434 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.892371 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 18 19:17:02.892601 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.892586 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jpm8c\"" Sep 18 19:17:02.893730 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.893715 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 18 19:17:02.897131 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.897102 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 18 19:17:02.914986 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.914964 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 18 19:17:02.922734 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.922710 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 19:17:02.939620 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939597 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.939739 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939626 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.939739 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939667 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.939739 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939686 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.939842 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939747 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.939842 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939795 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.939842 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939816 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.939842 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939835 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940016 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939854 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940016 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939883 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66hgm\" (UniqueName: \"kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-kube-api-access-66hgm\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940016 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939908 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940016 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939924 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940016 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.939985 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config-out\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940016 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.940003 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-web-config\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940016 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.940016 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.940080 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.940119 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.940170 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.940188 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:02.940325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:02.940202 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.040936 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.040908 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config-out\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.040951 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-web-config\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.040975 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041016 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041050 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041087 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041116 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041141 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041172 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041195 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041235 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041287 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041262 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041294 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041326 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041351 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041376 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041404 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041434 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66hgm\" (UniqueName: \"kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-kube-api-access-66hgm\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041486 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.041850 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.041513 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.042301 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.042273 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.044727 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.042996 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.044727 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.043482 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.044727 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.044068 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.044727 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.044363 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.045778 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.045733 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.047785 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.046990 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.050594 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.048407 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.050594 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.048930 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.050594 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.049381 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.050594 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.049909 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.050594 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.050089 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.050594 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.050102 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.050594 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.050375 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.051071 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.051014 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-web-config\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.051213 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.051173 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.051485 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.051444 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.052688 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.052495 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config-out\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.052688 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.052666 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.067000 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.066976 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66hgm\" (UniqueName: \"kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-kube-api-access-66hgm\") pod \"prometheus-k8s-0\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.165502 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.165461 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bf8d7d659-wz2zs"] Sep 18 19:17:03.181120 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.181085 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:03.392705 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.392670 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:03.493750 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.493662 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 19:17:03.497548 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:17:03.497519 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38285ffb_c4b8_4bcb_bfd2_486ba3d92c76.slice/crio-166339bdca3d699595ec2f540eea143b41e0b30fa853dd49ed2e7dc97dbece9a WatchSource:0}: Error finding container 166339bdca3d699595ec2f540eea143b41e0b30fa853dd49ed2e7dc97dbece9a: Status 404 returned error can't find the container with id 166339bdca3d699595ec2f540eea143b41e0b30fa853dd49ed2e7dc97dbece9a Sep 18 19:17:03.956633 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.956598 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" event={"ID":"3ae18d24-a8fa-4605-bd57-0fb210b92845","Type":"ContainerStarted","Data":"65f5c1e49190a5c80860eedb62077a40dd982cf83f52cb575de719a056aa16b3"} Sep 18 19:17:03.956842 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.956690 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" event={"ID":"3ae18d24-a8fa-4605-bd57-0fb210b92845","Type":"ContainerStarted","Data":"3db3fbae9216b7e46d002a0278d4c219a97cbe76f40d64278e292def1b9363f7"} Sep 18 19:17:03.956842 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.956735 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" event={"ID":"3ae18d24-a8fa-4605-bd57-0fb210b92845","Type":"ContainerStarted","Data":"9b916b9bb423a8bb0ff84414f125ea0be77c550142215b70ffa13969cf3b419d"} Sep 18 19:17:03.956842 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.956778 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:17:03.958431 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.958403 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" event={"ID":"6da8bc41-9fa5-4802-8f1f-08e96fed5f60","Type":"ContainerStarted","Data":"f8d9a3473cb305093040bb8e3e83bc6cef6d43cda0b5dc73a6a97ccbbb2d3e78"} Sep 18 19:17:03.958649 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.958599 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" Sep 18 19:17:03.959965 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.959941 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerStarted","Data":"166339bdca3d699595ec2f540eea143b41e0b30fa853dd49ed2e7dc97dbece9a"} Sep 18 19:17:03.964970 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:03.964951 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" Sep 18 19:17:04.028780 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:04.028714 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-t7kxg" podStartSLOduration=2.812318627 podStartE2EDuration="4.028690385s" totalImagesPullingTime="1.216371758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:17:04.027793303 +0000 UTC m=+169.932141063" watchObservedRunningTime="2026-09-18 19:17:04.028690385 +0000 UTC m=+169.933038142" Sep 18 19:17:04.029020 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:04.028993 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" podStartSLOduration=3.211435167 podStartE2EDuration="6.028983453s" totalImagesPullingTime="2.817548286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:17:03.998455769 +0000 UTC m=+169.902803529" watchObservedRunningTime="2026-09-18 19:17:04.028983453 +0000 UTC m=+169.933331211" Sep 18 19:17:04.967004 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:04.966974 2554 generic.go:356] "Generic (PLEG): container finished" podID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerID="09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585" exitCode=0 Sep 18 19:17:04.967488 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:04.967053 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerDied","Data":"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585"} Sep 18 19:17:05.936525 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:05.936490 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:06.055748 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:06.055712 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:08.271896 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:08.271717 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:17:08.987761 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:08.987726 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerStarted","Data":"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c"} Sep 18 19:17:08.987940 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:08.987813 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerStarted","Data":"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338"} Sep 18 19:17:08.987940 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:08.987841 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerStarted","Data":"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886"} Sep 18 19:17:08.987940 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:08.987863 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerStarted","Data":"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f"} Sep 18 19:17:08.987940 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:08.987886 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerStarted","Data":"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0"} Sep 18 19:17:08.987940 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:08.987912 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerStarted","Data":"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907"} Sep 18 19:17:09.938782 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:09.938748 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rbmm8" Sep 18 19:17:09.964022 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:09.963969 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.953152803 podStartE2EDuration="7.963950294s" totalImagesPullingTime="3.99198406s" totalInitContainerRuntime="18.813431ms" isStatefulPod=false podCreationTimestamp="2026-09-18 19:17:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:17:09.022021232 +0000 UTC m=+174.926369015" watchObservedRunningTime="2026-09-18 19:17:09.963950294 +0000 UTC m=+175.868298051" Sep 18 19:17:09.973344 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:09.973321 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-68665cdd46-v65g5" Sep 18 19:17:10.330708 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:10.330600 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:10.431681 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:10.431634 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:11.013654 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:11.013612 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:11.132347 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:11.132317 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:13.181362 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:13.181323 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:17:13.282426 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:13.282394 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:13.408438 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:13.408405 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:14.359199 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:14.359159 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:14.477270 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:14.477240 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:14.594735 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:14.594705 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:15.600759 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:15.600730 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:15.716995 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:15.716963 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:19.203615 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:19.203544 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:19.303291 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:19.303258 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:20.585703 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:20.585673 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:20.678725 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:20.678691 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:26.052564 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:26.052528 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54788fddc8-7b577_2cee06b4-79a2-47d4-9443-e1d1353ccc82/router/0.log" Sep 18 19:17:26.062077 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:26.062035 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n2xhf_879e230b-7166-438e-b7d4-e88e9116a54b/serve-healthcheck-canary/0.log" Sep 18 19:17:27.346589 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:27.346543 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:27.471199 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:27.471169 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:27.573599 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:27.573571 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:28.161501 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.161464 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:28.184417 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.184377 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bf8d7d659-wz2zs" podUID="ceaa43c6-d36b-4435-a11c-5c51981d9140" containerName="console" containerID="cri-o://642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863" gracePeriod=15 Sep 18 19:17:28.232245 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.232209 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:28.454158 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.454130 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:17:28.469093 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.469065 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-service-ca\") pod \"ceaa43c6-d36b-4435-a11c-5c51981d9140\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " Sep 18 19:17:28.469262 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.469107 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-config\") pod \"ceaa43c6-d36b-4435-a11c-5c51981d9140\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " Sep 18 19:17:28.469262 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.469153 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-serving-cert\") pod \"ceaa43c6-d36b-4435-a11c-5c51981d9140\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " Sep 18 19:17:28.469262 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.469182 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-oauth-serving-cert\") pod \"ceaa43c6-d36b-4435-a11c-5c51981d9140\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " Sep 18 19:17:28.469262 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.469214 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tn6p6\" (UniqueName: \"kubernetes.io/projected/ceaa43c6-d36b-4435-a11c-5c51981d9140-kube-api-access-tn6p6\") pod \"ceaa43c6-d36b-4435-a11c-5c51981d9140\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " Sep 18 19:17:28.469262 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.469258 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-oauth-config\") pod \"ceaa43c6-d36b-4435-a11c-5c51981d9140\" (UID: \"ceaa43c6-d36b-4435-a11c-5c51981d9140\") " Sep 18 19:17:28.469586 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.469553 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-service-ca" (OuterVolumeSpecName: "service-ca") pod "ceaa43c6-d36b-4435-a11c-5c51981d9140" (UID: "ceaa43c6-d36b-4435-a11c-5c51981d9140"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:17:28.472772 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.469967 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ceaa43c6-d36b-4435-a11c-5c51981d9140" (UID: "ceaa43c6-d36b-4435-a11c-5c51981d9140"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:17:28.472772 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.469962 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-config" (OuterVolumeSpecName: "console-config") pod "ceaa43c6-d36b-4435-a11c-5c51981d9140" (UID: "ceaa43c6-d36b-4435-a11c-5c51981d9140"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:17:28.472772 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.472258 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ceaa43c6-d36b-4435-a11c-5c51981d9140" (UID: "ceaa43c6-d36b-4435-a11c-5c51981d9140"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:17:28.472772 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.472277 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceaa43c6-d36b-4435-a11c-5c51981d9140-kube-api-access-tn6p6" (OuterVolumeSpecName: "kube-api-access-tn6p6") pod "ceaa43c6-d36b-4435-a11c-5c51981d9140" (UID: "ceaa43c6-d36b-4435-a11c-5c51981d9140"). InnerVolumeSpecName "kube-api-access-tn6p6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 19:17:28.472772 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.472284 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ceaa43c6-d36b-4435-a11c-5c51981d9140" (UID: "ceaa43c6-d36b-4435-a11c-5c51981d9140"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:17:28.571120 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.571081 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-service-ca\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:17:28.571120 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.571114 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-config\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:17:28.571120 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.571124 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-serving-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:17:28.571120 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.571133 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ceaa43c6-d36b-4435-a11c-5c51981d9140-oauth-serving-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:17:28.571386 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.571142 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tn6p6\" (UniqueName: \"kubernetes.io/projected/ceaa43c6-d36b-4435-a11c-5c51981d9140-kube-api-access-tn6p6\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:17:28.571386 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:28.571151 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ceaa43c6-d36b-4435-a11c-5c51981d9140-console-oauth-config\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:17:29.071185 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:29.071148 2554 generic.go:356] "Generic (PLEG): container finished" podID="ceaa43c6-d36b-4435-a11c-5c51981d9140" containerID="642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863" exitCode=0 Sep 18 19:17:29.071359 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:29.071200 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf8d7d659-wz2zs" Sep 18 19:17:29.071359 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:29.071227 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf8d7d659-wz2zs" event={"ID":"ceaa43c6-d36b-4435-a11c-5c51981d9140","Type":"ContainerDied","Data":"642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863"} Sep 18 19:17:29.071359 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:29.071300 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf8d7d659-wz2zs" event={"ID":"ceaa43c6-d36b-4435-a11c-5c51981d9140","Type":"ContainerDied","Data":"ec7d0a3f24b0831db8fa730575ddb5ba8c833697913142a9778af6c60853a1ac"} Sep 18 19:17:29.071359 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:29.071322 2554 scope.go:117] "RemoveContainer" containerID="642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863" Sep 18 19:17:29.088799 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:29.088779 2554 scope.go:117] "RemoveContainer" containerID="642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863" Sep 18 19:17:29.089109 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:17:29.089086 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863\": container with ID starting with 642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863 not found: ID does not exist" containerID="642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863" Sep 18 19:17:29.089181 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:29.089117 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863"} err="failed to get container status \"642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863\": rpc error: code = NotFound desc = could not find container \"642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863\": container with ID starting with 642b7b422bea8554f939976844268a55e999d02f05c1210f8c8897c77e736863 not found: ID does not exist" Sep 18 19:17:29.100738 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:29.100712 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bf8d7d659-wz2zs"] Sep 18 19:17:29.108233 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:29.108211 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bf8d7d659-wz2zs"] Sep 18 19:17:29.701860 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:29.701825 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:29.773865 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:29.773829 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:30.276695 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:30.276663 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceaa43c6-d36b-4435-a11c-5c51981d9140" path="/var/lib/kubelet/pods/ceaa43c6-d36b-4435-a11c-5c51981d9140/volumes" Sep 18 19:17:44.694984 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:44.694946 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:44.793363 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:44.793318 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:17:46.077164 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:17:46.077130 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:03.181932 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:03.181881 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:03.197326 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:03.197297 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:03.243040 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:03.243014 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:13.641259 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:13.641226 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:13.722338 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:13.722305 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:15.535372 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:15.535331 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:16.122752 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:16.122720 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:21.331768 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.331734 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 19:18:21.332242 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.332125 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="prometheus" containerID="cri-o://fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907" gracePeriod=600 Sep 18 19:18:21.332242 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.332145 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="config-reloader" containerID="cri-o://33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0" gracePeriod=600 Sep 18 19:18:21.332242 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.332146 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="thanos-sidecar" containerID="cri-o://9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f" gracePeriod=600 Sep 18 19:18:21.332242 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.332190 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy-thanos" containerID="cri-o://364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c" gracePeriod=600 Sep 18 19:18:21.332437 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.332304 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy-web" containerID="cri-o://2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886" gracePeriod=600 Sep 18 19:18:21.332437 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.332351 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy" containerID="cri-o://22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338" gracePeriod=600 Sep 18 19:18:21.577691 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.577667 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:21.721659 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.721619 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-kube-rbac-proxy\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.721659 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.721662 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-tls-assets\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.721916 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.721682 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-metrics-client-certs\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.721916 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.721791 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.721916 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.721832 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-trusted-ca-bundle\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.721916 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.721883 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.721916 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.721909 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-serving-certs-ca-bundle\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.722174 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.721938 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-db\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.722513 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722483 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.722616 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722529 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:18:21.722616 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722535 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-thanos-prometheus-http-client-file\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.722616 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722586 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-1\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.722793 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722614 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-kubelet-serving-ca-bundle\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.722793 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722765 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config-out\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.722883 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722813 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-2\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.722883 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722822 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:18:21.722883 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722857 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-grpc-tls\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.723039 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722882 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66hgm\" (UniqueName: \"kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-kube-api-access-66hgm\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.723039 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722917 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-0\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.723039 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722944 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-web-config\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.723039 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.722978 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-tls\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.723039 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.723019 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-metrics-client-ca\") pod \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\" (UID: \"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76\") " Sep 18 19:18:21.723280 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.723078 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:18:21.723334 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.723291 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.723334 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.723312 2554 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.723334 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.723329 2554 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.723482 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.723339 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:18:21.723597 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.723574 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:18:21.724023 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.723757 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 19:18:21.725810 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.724627 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:18:21.725810 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.724720 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:18:21.726316 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.726163 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:18:21.726316 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.726246 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config" (OuterVolumeSpecName: "config") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:18:21.726707 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.726669 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:18:21.727477 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.727449 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:18:21.727884 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.727839 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config-out" (OuterVolumeSpecName: "config-out") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 19:18:21.727961 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.727940 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:18:21.728195 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.728169 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:18:21.728278 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.728188 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:18:21.729836 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.729596 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 19:18:21.729836 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.729744 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-kube-api-access-66hgm" (OuterVolumeSpecName: "kube-api-access-66hgm") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "kube-api-access-66hgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 19:18:21.729836 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.729776 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:18:21.739125 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.739100 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-web-config" (OuterVolumeSpecName: "web-config") pod "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" (UID: "38285ffb-c4b8-4bcb-bfd2-486ba3d92c76"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:18:21.823918 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.823884 2554 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.823918 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.823915 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.823928 2554 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config-out\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.823941 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.823953 2554 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-grpc-tls\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.823964 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66hgm\" (UniqueName: \"kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-kube-api-access-66hgm\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.823977 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.823988 2554 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-web-config\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.824000 2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.824012 2554 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-configmap-metrics-client-ca\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.824029 2554 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-kube-rbac-proxy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.824040 2554 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-tls-assets\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.824053 2554 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-metrics-client-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.824064 2554 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-config\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.824077 2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.824089 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-prometheus-k8s-db\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:21.824220 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:21.824101 2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:18:22.297180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297147 2554 generic.go:356] "Generic (PLEG): container finished" podID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerID="364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c" exitCode=0 Sep 18 19:18:22.297180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297169 2554 generic.go:356] "Generic (PLEG): container finished" podID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerID="22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338" exitCode=0 Sep 18 19:18:22.297180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297176 2554 generic.go:356] "Generic (PLEG): container finished" podID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerID="2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886" exitCode=0 Sep 18 19:18:22.297180 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297184 2554 generic.go:356] "Generic (PLEG): container finished" podID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerID="9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f" exitCode=0 Sep 18 19:18:22.297425 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297192 2554 generic.go:356] "Generic (PLEG): container finished" podID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerID="33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0" exitCode=0 Sep 18 19:18:22.297425 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297201 2554 generic.go:356] "Generic (PLEG): container finished" podID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerID="fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907" exitCode=0 Sep 18 19:18:22.297425 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297200 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerDied","Data":"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c"} Sep 18 19:18:22.297425 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297221 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.297425 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297277 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerDied","Data":"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338"} Sep 18 19:18:22.297425 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297311 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerDied","Data":"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886"} Sep 18 19:18:22.297425 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297342 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerDied","Data":"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f"} Sep 18 19:18:22.297425 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297346 2554 scope.go:117] "RemoveContainer" containerID="364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c" Sep 18 19:18:22.297425 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297378 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerDied","Data":"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0"} Sep 18 19:18:22.297425 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297425 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerDied","Data":"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907"} Sep 18 19:18:22.297823 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.297463 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38285ffb-c4b8-4bcb-bfd2-486ba3d92c76","Type":"ContainerDied","Data":"166339bdca3d699595ec2f540eea143b41e0b30fa853dd49ed2e7dc97dbece9a"} Sep 18 19:18:22.305212 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.305192 2554 scope.go:117] "RemoveContainer" containerID="22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338" Sep 18 19:18:22.312754 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.312731 2554 scope.go:117] "RemoveContainer" containerID="2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886" Sep 18 19:18:22.319213 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.319198 2554 scope.go:117] "RemoveContainer" containerID="9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f" Sep 18 19:18:22.325602 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.325584 2554 scope.go:117] "RemoveContainer" containerID="33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0" Sep 18 19:18:22.332028 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.332010 2554 scope.go:117] "RemoveContainer" containerID="fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907" Sep 18 19:18:22.338796 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.338767 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 19:18:22.339981 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.339967 2554 scope.go:117] "RemoveContainer" containerID="09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585" Sep 18 19:18:22.348854 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.348825 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 19:18:22.348994 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.348879 2554 scope.go:117] "RemoveContainer" containerID="364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c" Sep 18 19:18:22.349399 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:18:22.349371 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": container with ID starting with 364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c not found: ID does not exist" containerID="364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c" Sep 18 19:18:22.349485 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.349408 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c"} err="failed to get container status \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": rpc error: code = NotFound desc = could not find container \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": container with ID starting with 364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c not found: ID does not exist" Sep 18 19:18:22.349485 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.349436 2554 scope.go:117] "RemoveContainer" containerID="22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338" Sep 18 19:18:22.349809 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:18:22.349770 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": container with ID starting with 22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338 not found: ID does not exist" containerID="22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338" Sep 18 19:18:22.349898 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.349819 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338"} err="failed to get container status \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": rpc error: code = NotFound desc = could not find container \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": container with ID starting with 22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338 not found: ID does not exist" Sep 18 19:18:22.349898 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.349843 2554 scope.go:117] "RemoveContainer" containerID="2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886" Sep 18 19:18:22.351455 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:18:22.351377 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": container with ID starting with 2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886 not found: ID does not exist" containerID="2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886" Sep 18 19:18:22.351455 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.351416 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886"} err="failed to get container status \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": rpc error: code = NotFound desc = could not find container \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": container with ID starting with 2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886 not found: ID does not exist" Sep 18 19:18:22.351455 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.351440 2554 scope.go:117] "RemoveContainer" containerID="9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f" Sep 18 19:18:22.351982 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:18:22.351831 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": container with ID starting with 9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f not found: ID does not exist" containerID="9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f" Sep 18 19:18:22.351982 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.351876 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f"} err="failed to get container status \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": rpc error: code = NotFound desc = could not find container \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": container with ID starting with 9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f not found: ID does not exist" Sep 18 19:18:22.351982 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.351899 2554 scope.go:117] "RemoveContainer" containerID="33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0" Sep 18 19:18:22.352578 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:18:22.352550 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": container with ID starting with 33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0 not found: ID does not exist" containerID="33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0" Sep 18 19:18:22.352704 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.352586 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0"} err="failed to get container status \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": rpc error: code = NotFound desc = could not find container \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": container with ID starting with 33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0 not found: ID does not exist" Sep 18 19:18:22.352704 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.352606 2554 scope.go:117] "RemoveContainer" containerID="fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907" Sep 18 19:18:22.352979 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:18:22.352953 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": container with ID starting with fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907 not found: ID does not exist" containerID="fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907" Sep 18 19:18:22.353078 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.352986 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907"} err="failed to get container status \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": rpc error: code = NotFound desc = could not find container \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": container with ID starting with fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907 not found: ID does not exist" Sep 18 19:18:22.353078 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.353008 2554 scope.go:117] "RemoveContainer" containerID="09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585" Sep 18 19:18:22.353312 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:18:22.353292 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": container with ID starting with 09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585 not found: ID does not exist" containerID="09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585" Sep 18 19:18:22.353399 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.353321 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585"} err="failed to get container status \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": rpc error: code = NotFound desc = could not find container \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": container with ID starting with 09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585 not found: ID does not exist" Sep 18 19:18:22.353399 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.353343 2554 scope.go:117] "RemoveContainer" containerID="364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c" Sep 18 19:18:22.353702 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.353673 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c"} err="failed to get container status \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": rpc error: code = NotFound desc = could not find container \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": container with ID starting with 364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c not found: ID does not exist" Sep 18 19:18:22.353799 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.353704 2554 scope.go:117] "RemoveContainer" containerID="22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338" Sep 18 19:18:22.354006 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.353983 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338"} err="failed to get container status \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": rpc error: code = NotFound desc = could not find container \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": container with ID starting with 22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338 not found: ID does not exist" Sep 18 19:18:22.354006 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.354004 2554 scope.go:117] "RemoveContainer" containerID="2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886" Sep 18 19:18:22.354274 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.354252 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886"} err="failed to get container status \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": rpc error: code = NotFound desc = could not find container \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": container with ID starting with 2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886 not found: ID does not exist" Sep 18 19:18:22.354356 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.354274 2554 scope.go:117] "RemoveContainer" containerID="9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f" Sep 18 19:18:22.354551 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.354528 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f"} err="failed to get container status \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": rpc error: code = NotFound desc = could not find container \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": container with ID starting with 9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f not found: ID does not exist" Sep 18 19:18:22.354630 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.354551 2554 scope.go:117] "RemoveContainer" containerID="33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0" Sep 18 19:18:22.354865 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.354845 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0"} err="failed to get container status \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": rpc error: code = NotFound desc = could not find container \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": container with ID starting with 33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0 not found: ID does not exist" Sep 18 19:18:22.354948 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.354877 2554 scope.go:117] "RemoveContainer" containerID="fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907" Sep 18 19:18:22.355150 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.355120 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907"} err="failed to get container status \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": rpc error: code = NotFound desc = could not find container \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": container with ID starting with fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907 not found: ID does not exist" Sep 18 19:18:22.355150 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.355146 2554 scope.go:117] "RemoveContainer" containerID="09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585" Sep 18 19:18:22.355468 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.355442 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585"} err="failed to get container status \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": rpc error: code = NotFound desc = could not find container \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": container with ID starting with 09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585 not found: ID does not exist" Sep 18 19:18:22.355560 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.355468 2554 scope.go:117] "RemoveContainer" containerID="364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c" Sep 18 19:18:22.355752 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.355730 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c"} err="failed to get container status \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": rpc error: code = NotFound desc = could not find container \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": container with ID starting with 364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c not found: ID does not exist" Sep 18 19:18:22.355819 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.355752 2554 scope.go:117] "RemoveContainer" containerID="22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338" Sep 18 19:18:22.355993 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.355976 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338"} err="failed to get container status \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": rpc error: code = NotFound desc = could not find container \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": container with ID starting with 22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338 not found: ID does not exist" Sep 18 19:18:22.356112 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.355993 2554 scope.go:117] "RemoveContainer" containerID="2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886" Sep 18 19:18:22.356237 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.356221 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886"} err="failed to get container status \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": rpc error: code = NotFound desc = could not find container \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": container with ID starting with 2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886 not found: ID does not exist" Sep 18 19:18:22.356286 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.356237 2554 scope.go:117] "RemoveContainer" containerID="9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f" Sep 18 19:18:22.356484 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.356467 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f"} err="failed to get container status \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": rpc error: code = NotFound desc = could not find container \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": container with ID starting with 9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f not found: ID does not exist" Sep 18 19:18:22.356542 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.356484 2554 scope.go:117] "RemoveContainer" containerID="33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0" Sep 18 19:18:22.356746 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.356726 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0"} err="failed to get container status \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": rpc error: code = NotFound desc = could not find container \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": container with ID starting with 33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0 not found: ID does not exist" Sep 18 19:18:22.356810 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.356747 2554 scope.go:117] "RemoveContainer" containerID="fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907" Sep 18 19:18:22.357029 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.357006 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907"} err="failed to get container status \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": rpc error: code = NotFound desc = could not find container \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": container with ID starting with fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907 not found: ID does not exist" Sep 18 19:18:22.357029 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.357029 2554 scope.go:117] "RemoveContainer" containerID="09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585" Sep 18 19:18:22.357280 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.357261 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585"} err="failed to get container status \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": rpc error: code = NotFound desc = could not find container \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": container with ID starting with 09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585 not found: ID does not exist" Sep 18 19:18:22.357351 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.357291 2554 scope.go:117] "RemoveContainer" containerID="364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c" Sep 18 19:18:22.357534 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.357517 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c"} err="failed to get container status \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": rpc error: code = NotFound desc = could not find container \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": container with ID starting with 364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c not found: ID does not exist" Sep 18 19:18:22.357587 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.357534 2554 scope.go:117] "RemoveContainer" containerID="22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338" Sep 18 19:18:22.357774 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.357753 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338"} err="failed to get container status \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": rpc error: code = NotFound desc = could not find container \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": container with ID starting with 22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338 not found: ID does not exist" Sep 18 19:18:22.357859 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.357775 2554 scope.go:117] "RemoveContainer" containerID="2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886" Sep 18 19:18:22.358073 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.358051 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886"} err="failed to get container status \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": rpc error: code = NotFound desc = could not find container \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": container with ID starting with 2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886 not found: ID does not exist" Sep 18 19:18:22.358130 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.358074 2554 scope.go:117] "RemoveContainer" containerID="9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f" Sep 18 19:18:22.358324 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.358306 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f"} err="failed to get container status \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": rpc error: code = NotFound desc = could not find container \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": container with ID starting with 9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f not found: ID does not exist" Sep 18 19:18:22.358379 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.358326 2554 scope.go:117] "RemoveContainer" containerID="33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0" Sep 18 19:18:22.358579 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.358559 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0"} err="failed to get container status \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": rpc error: code = NotFound desc = could not find container \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": container with ID starting with 33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0 not found: ID does not exist" Sep 18 19:18:22.358579 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.358577 2554 scope.go:117] "RemoveContainer" containerID="fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907" Sep 18 19:18:22.358827 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.358812 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907"} err="failed to get container status \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": rpc error: code = NotFound desc = could not find container \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": container with ID starting with fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907 not found: ID does not exist" Sep 18 19:18:22.358827 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.358826 2554 scope.go:117] "RemoveContainer" containerID="09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585" Sep 18 19:18:22.359109 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.359091 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585"} err="failed to get container status \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": rpc error: code = NotFound desc = could not find container \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": container with ID starting with 09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585 not found: ID does not exist" Sep 18 19:18:22.359170 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.359110 2554 scope.go:117] "RemoveContainer" containerID="364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c" Sep 18 19:18:22.359371 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.359353 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c"} err="failed to get container status \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": rpc error: code = NotFound desc = could not find container \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": container with ID starting with 364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c not found: ID does not exist" Sep 18 19:18:22.359371 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.359371 2554 scope.go:117] "RemoveContainer" containerID="22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338" Sep 18 19:18:22.359606 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.359590 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338"} err="failed to get container status \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": rpc error: code = NotFound desc = could not find container \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": container with ID starting with 22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338 not found: ID does not exist" Sep 18 19:18:22.359606 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.359607 2554 scope.go:117] "RemoveContainer" containerID="2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886" Sep 18 19:18:22.359873 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.359849 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886"} err="failed to get container status \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": rpc error: code = NotFound desc = could not find container \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": container with ID starting with 2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886 not found: ID does not exist" Sep 18 19:18:22.359923 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.359876 2554 scope.go:117] "RemoveContainer" containerID="9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f" Sep 18 19:18:22.360121 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.360103 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f"} err="failed to get container status \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": rpc error: code = NotFound desc = could not find container \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": container with ID starting with 9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f not found: ID does not exist" Sep 18 19:18:22.360159 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.360121 2554 scope.go:117] "RemoveContainer" containerID="33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0" Sep 18 19:18:22.360364 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.360345 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0"} err="failed to get container status \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": rpc error: code = NotFound desc = could not find container \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": container with ID starting with 33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0 not found: ID does not exist" Sep 18 19:18:22.360435 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.360365 2554 scope.go:117] "RemoveContainer" containerID="fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907" Sep 18 19:18:22.360660 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.360620 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907"} err="failed to get container status \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": rpc error: code = NotFound desc = could not find container \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": container with ID starting with fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907 not found: ID does not exist" Sep 18 19:18:22.360660 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.360657 2554 scope.go:117] "RemoveContainer" containerID="09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585" Sep 18 19:18:22.360935 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.360917 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585"} err="failed to get container status \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": rpc error: code = NotFound desc = could not find container \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": container with ID starting with 09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585 not found: ID does not exist" Sep 18 19:18:22.360935 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.360935 2554 scope.go:117] "RemoveContainer" containerID="364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c" Sep 18 19:18:22.361193 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.361175 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c"} err="failed to get container status \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": rpc error: code = NotFound desc = could not find container \"364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c\": container with ID starting with 364294eac105b3b2e0e6e0297ecd9d02c012f4789d6bdc0000e77f61b0f8252c not found: ID does not exist" Sep 18 19:18:22.361193 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.361192 2554 scope.go:117] "RemoveContainer" containerID="22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338" Sep 18 19:18:22.361445 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.361426 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338"} err="failed to get container status \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": rpc error: code = NotFound desc = could not find container \"22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338\": container with ID starting with 22403b41c36360bd29b6099cf9583d47e9caf2db9e86c8b7bace04782cf31338 not found: ID does not exist" Sep 18 19:18:22.361510 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.361445 2554 scope.go:117] "RemoveContainer" containerID="2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886" Sep 18 19:18:22.361736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.361718 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886"} err="failed to get container status \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": rpc error: code = NotFound desc = could not find container \"2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886\": container with ID starting with 2d5809212e4758fa2ba285af1152ac6f0cce25f55d21e0c584a2534607c1b886 not found: ID does not exist" Sep 18 19:18:22.361736 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.361736 2554 scope.go:117] "RemoveContainer" containerID="9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f" Sep 18 19:18:22.361959 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.361942 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f"} err="failed to get container status \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": rpc error: code = NotFound desc = could not find container \"9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f\": container with ID starting with 9435f6370dcf3583ec793db03a0ff3a522b04037aa7469208c0069c5a86a789f not found: ID does not exist" Sep 18 19:18:22.362008 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.361959 2554 scope.go:117] "RemoveContainer" containerID="33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0" Sep 18 19:18:22.362154 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.362139 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0"} err="failed to get container status \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": rpc error: code = NotFound desc = could not find container \"33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0\": container with ID starting with 33374a873fcb5935d3f2c549c569e1364b100da32997518b1f6c482958da33d0 not found: ID does not exist" Sep 18 19:18:22.362192 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.362154 2554 scope.go:117] "RemoveContainer" containerID="fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907" Sep 18 19:18:22.362351 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.362336 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907"} err="failed to get container status \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": rpc error: code = NotFound desc = could not find container \"fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907\": container with ID starting with fdda34a3903189361d6966563bf74d27dfb76d7ec8298b69f2065f572648f907 not found: ID does not exist" Sep 18 19:18:22.362410 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.362351 2554 scope.go:117] "RemoveContainer" containerID="09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585" Sep 18 19:18:22.362544 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.362528 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585"} err="failed to get container status \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": rpc error: code = NotFound desc = could not find container \"09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585\": container with ID starting with 09dd1e6325d25d932d01340f213b7d5130be2fb1adad5f2bdbb9170e584a1585 not found: ID does not exist" Sep 18 19:18:22.402683 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.402650 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 19:18:22.402993 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.402978 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="config-reloader" Sep 18 19:18:22.403066 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.402996 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="config-reloader" Sep 18 19:18:22.403066 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403009 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="prometheus" Sep 18 19:18:22.403066 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403016 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="prometheus" Sep 18 19:18:22.403066 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403029 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceaa43c6-d36b-4435-a11c-5c51981d9140" containerName="console" Sep 18 19:18:22.403066 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403038 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceaa43c6-d36b-4435-a11c-5c51981d9140" containerName="console" Sep 18 19:18:22.403066 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403050 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="thanos-sidecar" Sep 18 19:18:22.403066 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403059 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="thanos-sidecar" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403068 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403077 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403089 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy-thanos" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403098 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy-thanos" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403109 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy-web" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403118 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy-web" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403135 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="init-config-reloader" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403144 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="init-config-reloader" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403221 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="config-reloader" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403231 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy-web" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403240 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="thanos-sidecar" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403248 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy-thanos" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403256 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="kube-rbac-proxy" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403265 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" containerName="prometheus" Sep 18 19:18:22.403394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.403275 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceaa43c6-d36b-4435-a11c-5c51981d9140" containerName="console" Sep 18 19:18:22.408927 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.408908 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.411754 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.411737 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 18 19:18:22.411863 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.411849 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 18 19:18:22.412167 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.412151 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 18 19:18:22.412401 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.412387 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 18 19:18:22.412772 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.412757 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b8bllk14hahl5\"" Sep 18 19:18:22.413741 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.413717 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 18 19:18:22.414761 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.414742 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 18 19:18:22.421392 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.421377 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 18 19:18:22.421694 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.421680 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 18 19:18:22.421794 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.421771 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 18 19:18:22.421866 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.421854 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 18 19:18:22.421932 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.421866 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 18 19:18:22.422075 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.422055 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jpm8c\"" Sep 18 19:18:22.422151 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.422131 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 18 19:18:22.422214 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.422148 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 18 19:18:22.422555 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.422537 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 18 19:18:22.423885 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.423870 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 18 19:18:22.440481 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.440452 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 19:18:22.531091 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531064 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531091 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531093 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531318 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531121 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531318 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531143 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-web-config\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531318 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531189 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531318 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531233 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531318 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531257 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531318 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531287 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2wt8\" (UniqueName: \"kubernetes.io/projected/c4b39b77-a033-49f0-a606-59152138a433-kube-api-access-b2wt8\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531318 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531309 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531329 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-config\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531349 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531372 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531390 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531405 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531421 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4b39b77-a033-49f0-a606-59152138a433-config-out\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531442 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531457 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531482 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531527 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531514 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.531824 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.531544 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4b39b77-a033-49f0-a606-59152138a433-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632038 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632005 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2wt8\" (UniqueName: \"kubernetes.io/projected/c4b39b77-a033-49f0-a606-59152138a433-kube-api-access-b2wt8\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632175 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632047 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632175 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632067 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-config\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632257 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632184 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632257 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632219 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632257 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632250 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632406 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632277 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632406 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632304 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4b39b77-a033-49f0-a606-59152138a433-config-out\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632406 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632333 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632406 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632358 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632406 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632400 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632710 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632428 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632710 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632469 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4b39b77-a033-49f0-a606-59152138a433-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632710 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632497 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632710 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632520 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632710 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632570 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632710 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632603 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-web-config\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632710 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632628 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.632710 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632629 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.633170 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632740 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.633170 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632772 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.633170 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.632863 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.633325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.633274 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.633438 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.633412 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.635363 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.635149 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-config\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.635363 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.635322 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.635552 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.635454 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.635552 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.635463 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.636294 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.635989 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.636294 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.636019 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4b39b77-a033-49f0-a606-59152138a433-config-out\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.636294 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.636019 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4b39b77-a033-49f0-a606-59152138a433-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.636294 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.636058 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.636553 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.636325 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.636702 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.636678 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.637125 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.637106 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c4b39b77-a033-49f0-a606-59152138a433-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.638157 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.638136 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.638265 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.638244 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-web-config\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.638307 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.638252 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.638677 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.638661 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c4b39b77-a033-49f0-a606-59152138a433-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.641511 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.641488 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2wt8\" (UniqueName: \"kubernetes.io/projected/c4b39b77-a033-49f0-a606-59152138a433-kube-api-access-b2wt8\") pod \"prometheus-k8s-0\" (UID: \"c4b39b77-a033-49f0-a606-59152138a433\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:22.719911 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:22.719869 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:23.021701 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:23.021594 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 19:18:23.026070 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:18:23.026040 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b39b77_a033_49f0_a606_59152138a433.slice/crio-ad266a917f0d1c8fa44d43fade400b781a390dbd8d588e650c7aef0aa0a78ee6 WatchSource:0}: Error finding container ad266a917f0d1c8fa44d43fade400b781a390dbd8d588e650c7aef0aa0a78ee6: Status 404 returned error can't find the container with id ad266a917f0d1c8fa44d43fade400b781a390dbd8d588e650c7aef0aa0a78ee6 Sep 18 19:18:23.303376 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:23.303299 2554 generic.go:356] "Generic (PLEG): container finished" podID="c4b39b77-a033-49f0-a606-59152138a433" containerID="b57562abfd3731412244df744671f55bb32814bfe102ed0656a1c7f87e2f61be" exitCode=0 Sep 18 19:18:23.303376 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:23.303348 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4b39b77-a033-49f0-a606-59152138a433","Type":"ContainerDied","Data":"b57562abfd3731412244df744671f55bb32814bfe102ed0656a1c7f87e2f61be"} Sep 18 19:18:23.303547 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:23.303397 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4b39b77-a033-49f0-a606-59152138a433","Type":"ContainerStarted","Data":"ad266a917f0d1c8fa44d43fade400b781a390dbd8d588e650c7aef0aa0a78ee6"} Sep 18 19:18:24.277035 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:24.276998 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38285ffb-c4b8-4bcb-bfd2-486ba3d92c76" path="/var/lib/kubelet/pods/38285ffb-c4b8-4bcb-bfd2-486ba3d92c76/volumes" Sep 18 19:18:24.310324 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:24.310289 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4b39b77-a033-49f0-a606-59152138a433","Type":"ContainerStarted","Data":"2f1e0f87de8bc2dd9346d09ae3af396c2f5d8c7d1479bcd78ee44a305daf2b08"} Sep 18 19:18:24.310446 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:24.310356 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4b39b77-a033-49f0-a606-59152138a433","Type":"ContainerStarted","Data":"e7ffbdc6525c660bf79fb8b24914e7ddc3a4f3e185ee6012fa39474081e1929b"} Sep 18 19:18:24.310446 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:24.310389 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4b39b77-a033-49f0-a606-59152138a433","Type":"ContainerStarted","Data":"13facc4e138fe34f9dd8b73ea1dfb92f00029debc1b26727084be79c634ca10b"} Sep 18 19:18:24.310446 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:24.310417 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4b39b77-a033-49f0-a606-59152138a433","Type":"ContainerStarted","Data":"140dce21dc21565627541a234dc6281c2d8b1d6d3892117b312f77746754432a"} Sep 18 19:18:24.310446 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:24.310441 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4b39b77-a033-49f0-a606-59152138a433","Type":"ContainerStarted","Data":"9619d3e088cdac46007e225491513fd4f01fb2bf45e3840752d90a712873e7c9"} Sep 18 19:18:24.310596 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:24.310475 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4b39b77-a033-49f0-a606-59152138a433","Type":"ContainerStarted","Data":"f5883f89c43170f001b413fe3757d2b148d45fba18ff37c3396ec4ad1b512952"} Sep 18 19:18:24.366550 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:24.366498 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.3468629930000002 podStartE2EDuration="2.366481144s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.618151ms" isStatefulPod=false podCreationTimestamp="2026-09-18 19:18:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:18:24.366114257 +0000 UTC m=+250.270462041" watchObservedRunningTime="2026-09-18 19:18:24.366481144 +0000 UTC m=+250.270828902" Sep 18 19:18:24.825652 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:24.825615 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:24.922248 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:24.922214 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:27.720757 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:27.720681 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:18:27.985281 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:27.985182 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:18:27.987436 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:27.987416 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0899c67e-8d96-4fa0-b310-b6641c91127b-metrics-certs\") pod \"network-metrics-daemon-dvkb2\" (UID: \"0899c67e-8d96-4fa0-b310-b6641c91127b\") " pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:18:28.075306 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:28.075266 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w2qlj\"" Sep 18 19:18:28.083036 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:28.083007 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvkb2" Sep 18 19:18:28.374983 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:28.374958 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dvkb2"] Sep 18 19:18:28.377258 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:18:28.377234 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0899c67e_8d96_4fa0_b310_b6641c91127b.slice/crio-4cbdf2e10097e59d918249b10192c18dfffbc5950c7cd1f7d57d2a3ada56ece8 WatchSource:0}: Error finding container 4cbdf2e10097e59d918249b10192c18dfffbc5950c7cd1f7d57d2a3ada56ece8: Status 404 returned error can't find the container with id 4cbdf2e10097e59d918249b10192c18dfffbc5950c7cd1f7d57d2a3ada56ece8 Sep 18 19:18:29.331038 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:29.331009 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvkb2" event={"ID":"0899c67e-8d96-4fa0-b310-b6641c91127b","Type":"ContainerStarted","Data":"4cbdf2e10097e59d918249b10192c18dfffbc5950c7cd1f7d57d2a3ada56ece8"} Sep 18 19:18:30.336517 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:30.336485 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvkb2" event={"ID":"0899c67e-8d96-4fa0-b310-b6641c91127b","Type":"ContainerStarted","Data":"476fcaf013f9f96e57293adfac24c118b15842a1c90757fb5b05d58b65b592cf"} Sep 18 19:18:30.336925 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:30.336532 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvkb2" event={"ID":"0899c67e-8d96-4fa0-b310-b6641c91127b","Type":"ContainerStarted","Data":"706150c25b5cf866c8e1ab92005ac36fac50aac5240531d909f0778f8ca4e61b"} Sep 18 19:18:30.359250 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:30.359195 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dvkb2" podStartSLOduration=253.488090918 podStartE2EDuration="4m14.359172988s" totalImagesPullingTime="871.08207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:18:30.358875479 +0000 UTC m=+256.263223251" watchObservedRunningTime="2026-09-18 19:18:30.359172988 +0000 UTC m=+256.263520745" Sep 18 19:18:30.519672 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:30.519624 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:30.597474 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:30.597380 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:33.679696 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:33.679662 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:33.768974 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:33.768936 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:35.815156 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:35.815118 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:35.914432 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:35.914402 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:35.992485 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:35.992453 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:53.269628 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:53.269594 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:53.369959 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:53.369923 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:56.118719 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:56.118684 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:18:56.185024 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:18:56.184990 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:16.112119 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:16.112090 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:19:16.112943 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:16.112919 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:19:16.118948 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:16.118932 2554 kubelet.go:1648] "Image garbage collection succeeded" Sep 18 19:19:22.720999 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:22.720961 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:19:22.736136 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:22.736114 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:19:23.547047 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:23.547023 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 19:19:31.073592 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:31.073557 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:31.175554 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:31.175520 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:32.467598 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:32.467545 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:32.564530 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:32.564501 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:32.665489 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:32.665457 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:33.197491 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:33.197460 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:33.266275 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:33.266248 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:43.022555 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:43.022525 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:43.118250 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:43.118219 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:43.218857 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:43.218828 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:48.089699 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:48.089662 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:59.825754 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:59.825676 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:19:59.935867 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:19:59.935837 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:20:00.034829 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:20:00.034800 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:20:05.509285 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:20:05.509243 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:20:47.441173 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:20:47.441141 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:20:47.576908 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:20:47.576878 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:20:47.695916 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:20:47.695842 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:21:10.440521 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:21:10.440483 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:21:10.620733 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:21:10.620696 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:21:17.880415 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:21:17.880376 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:21:19.377805 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:21:19.377769 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:21:19.704566 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:21:19.704533 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:21:20.131950 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:21:20.131924 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:21:20.575318 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:21:20.575241 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:21:20.712417 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:21:20.712385 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:21:58.119474 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:21:58.119435 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:21:58.203719 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:21:58.203686 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:09.531394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:09.531361 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:09.624086 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:09.624055 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:09.828063 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:09.827986 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:09.930714 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:09.930686 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:44.962972 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:44.962939 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:45.038881 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:45.038853 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:46.088499 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:46.088471 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:46.171322 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:46.171289 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:46.288588 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:46.288556 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:46.396900 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:46.396861 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:46.549326 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:46.549294 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:46.888182 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:46.888149 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:46.987543 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:46.987509 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:22:47.080956 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:22:47.080919 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:24:07.350375 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:07.350332 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:24:07.471961 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:07.471929 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:24:07.580352 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:07.580319 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:24:07.769356 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:07.769326 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:24:07.911208 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:07.911165 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:24:09.938261 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:09.938231 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:24:10.011914 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:10.011866 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:24:16.144392 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:16.144359 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:24:16.144879 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:16.144670 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:24:24.541196 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.541154 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-8flxs"] Sep 18 19:24:24.548174 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.545283 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-8flxs" Sep 18 19:24:24.548374 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.548342 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-fvf6p\"" Sep 18 19:24:24.548487 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.548387 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Sep 18 19:24:24.548842 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.548825 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\"" Sep 18 19:24:24.549028 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.549010 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Sep 18 19:24:24.551494 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.551479 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Sep 18 19:24:24.557753 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.557732 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-8flxs"] Sep 18 19:24:24.680325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.680300 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj5zq\" (UniqueName: \"kubernetes.io/projected/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-kube-api-access-nj5zq\") pod \"s3-init-8flxs\" (UID: \"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958\") " pod="kserve/s3-init-8flxs" Sep 18 19:24:24.680478 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.680343 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-model-cache\") pod \"s3-init-8flxs\" (UID: \"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958\") " pod="kserve/s3-init-8flxs" Sep 18 19:24:24.781387 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.781357 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj5zq\" (UniqueName: \"kubernetes.io/projected/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-kube-api-access-nj5zq\") pod \"s3-init-8flxs\" (UID: \"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958\") " pod="kserve/s3-init-8flxs" Sep 18 19:24:24.781536 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.781408 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-model-cache\") pod \"s3-init-8flxs\" (UID: \"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958\") " pod="kserve/s3-init-8flxs" Sep 18 19:24:24.781751 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.781736 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-model-cache\") pod \"s3-init-8flxs\" (UID: \"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958\") " pod="kserve/s3-init-8flxs" Sep 18 19:24:24.791551 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.791510 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj5zq\" (UniqueName: \"kubernetes.io/projected/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-kube-api-access-nj5zq\") pod \"s3-init-8flxs\" (UID: \"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958\") " pod="kserve/s3-init-8flxs" Sep 18 19:24:24.866098 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:24.866054 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-8flxs" Sep 18 19:24:25.144019 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:25.143987 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-8flxs"] Sep 18 19:24:25.147134 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:24:25.147108 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbeb0e0e_5bc3_4b98_980d_56f7f5f03958.slice/crio-4db47a68cca1301bc4ed546456f214fdaa0fe87cbc74ab6eb74348fd980d7b4e WatchSource:0}: Error finding container 4db47a68cca1301bc4ed546456f214fdaa0fe87cbc74ab6eb74348fd980d7b4e: Status 404 returned error can't find the container with id 4db47a68cca1301bc4ed546456f214fdaa0fe87cbc74ab6eb74348fd980d7b4e Sep 18 19:24:25.149028 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:25.149011 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 19:24:25.669478 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:25.669443 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8flxs" event={"ID":"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958","Type":"ContainerStarted","Data":"4db47a68cca1301bc4ed546456f214fdaa0fe87cbc74ab6eb74348fd980d7b4e"} Sep 18 19:24:28.684774 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:28.684692 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8flxs" event={"ID":"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958","Type":"ContainerStarted","Data":"37a0ceb8847368ae16195a2eb0c7f1e9fc9594e3b8dc9e1739542b71e21b6ffe"} Sep 18 19:24:31.698819 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:31.698789 2554 generic.go:356] "Generic (PLEG): container finished" podID="cbeb0e0e-5bc3-4b98-980d-56f7f5f03958" containerID="37a0ceb8847368ae16195a2eb0c7f1e9fc9594e3b8dc9e1739542b71e21b6ffe" exitCode=0 Sep 18 19:24:31.699175 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:31.698861 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8flxs" event={"ID":"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958","Type":"ContainerDied","Data":"37a0ceb8847368ae16195a2eb0c7f1e9fc9594e3b8dc9e1739542b71e21b6ffe"} Sep 18 19:24:36.724986 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:36.724953 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8flxs" event={"ID":"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958","Type":"ContainerStarted","Data":"c0919b49d897d94900ed40dd9a37f0dd9242978b9876ecef24d3092e50340e17"} Sep 18 19:24:36.754228 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:36.754181 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-8flxs" podStartSLOduration=2.761389475 podStartE2EDuration="12.754165558s" totalImagesPullingTime="7.756006518s" totalInitContainerRuntime="2.236769565s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:24:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:24:36.753680812 +0000 UTC m=+622.658028568" watchObservedRunningTime="2026-09-18 19:24:36.754165558 +0000 UTC m=+622.658513315" Sep 18 19:24:41.173184 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:24:41.173137 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:08.210110 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:08.210066 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:08.280972 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:08.280946 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:41.973652 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:41.973605 2554 generic.go:356] "Generic (PLEG): container finished" podID="cbeb0e0e-5bc3-4b98-980d-56f7f5f03958" containerID="c0919b49d897d94900ed40dd9a37f0dd9242978b9876ecef24d3092e50340e17" exitCode=0 Sep 18 19:25:41.974054 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:41.973691 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8flxs" event={"ID":"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958","Type":"ContainerDied","Data":"c0919b49d897d94900ed40dd9a37f0dd9242978b9876ecef24d3092e50340e17"} Sep 18 19:25:43.099215 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:43.099190 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-8flxs" Sep 18 19:25:43.117946 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:43.117921 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-model-cache\") pod \"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958\" (UID: \"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958\") " Sep 18 19:25:43.118040 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:43.117974 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj5zq\" (UniqueName: \"kubernetes.io/projected/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-kube-api-access-nj5zq\") pod \"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958\" (UID: \"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958\") " Sep 18 19:25:43.120382 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:43.120350 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-kube-api-access-nj5zq" (OuterVolumeSpecName: "kube-api-access-nj5zq") pod "cbeb0e0e-5bc3-4b98-980d-56f7f5f03958" (UID: "cbeb0e0e-5bc3-4b98-980d-56f7f5f03958"). InnerVolumeSpecName "kube-api-access-nj5zq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 19:25:43.148494 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:43.148453 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-model-cache" (OuterVolumeSpecName: "model-cache") pod "cbeb0e0e-5bc3-4b98-980d-56f7f5f03958" (UID: "cbeb0e0e-5bc3-4b98-980d-56f7f5f03958"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 19:25:43.218784 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:43.218742 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-model-cache\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:25:43.218784 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:43.218776 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj5zq\" (UniqueName: \"kubernetes.io/projected/cbeb0e0e-5bc3-4b98-980d-56f7f5f03958-kube-api-access-nj5zq\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:25:43.982166 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:43.982131 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8flxs" event={"ID":"cbeb0e0e-5bc3-4b98-980d-56f7f5f03958","Type":"ContainerDied","Data":"4db47a68cca1301bc4ed546456f214fdaa0fe87cbc74ab6eb74348fd980d7b4e"} Sep 18 19:25:43.982166 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:43.982173 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4db47a68cca1301bc4ed546456f214fdaa0fe87cbc74ab6eb74348fd980d7b4e" Sep 18 19:25:43.982365 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:43.982139 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-8flxs" Sep 18 19:25:44.358161 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:44.358008 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:44.459134 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:44.459097 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:56.516104 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.516068 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:56.645431 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.645398 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz"] Sep 18 19:25:56.645714 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.645700 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbeb0e0e-5bc3-4b98-980d-56f7f5f03958" containerName="download-hf-model" Sep 18 19:25:56.645714 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.645714 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbeb0e0e-5bc3-4b98-980d-56f7f5f03958" containerName="download-hf-model" Sep 18 19:25:56.645827 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.645721 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbeb0e0e-5bc3-4b98-980d-56f7f5f03958" containerName="s3-init" Sep 18 19:25:56.645827 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.645727 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbeb0e0e-5bc3-4b98-980d-56f7f5f03958" containerName="s3-init" Sep 18 19:25:56.645827 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.645789 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbeb0e0e-5bc3-4b98-980d-56f7f5f03958" containerName="s3-init" Sep 18 19:25:56.648981 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.648965 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.651101 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.651075 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Sep 18 19:25:56.651227 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.651075 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 18 19:25:56.651486 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.651468 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Sep 18 19:25:56.651593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.651467 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Sep 18 19:25:56.651593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.651508 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Sep 18 19:25:56.651593 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.651565 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-qfhnz\"" Sep 18 19:25:56.651878 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.651632 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Sep 18 19:25:56.659481 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.659462 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz"] Sep 18 19:25:56.716150 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.716118 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:56.728197 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.728174 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.728390 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.728211 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78194fb5-9251-42bd-bba8-2849f2223f5f-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.728390 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.728237 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-447l8\" (UniqueName: \"kubernetes.io/projected/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-api-access-447l8\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.728390 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.728269 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.728390 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.728316 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78194fb5-9251-42bd-bba8-2849f2223f5f-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.828952 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.828861 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78194fb5-9251-42bd-bba8-2849f2223f5f-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.828952 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.828926 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.829177 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.828971 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78194fb5-9251-42bd-bba8-2849f2223f5f-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.829177 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.829009 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-447l8\" (UniqueName: \"kubernetes.io/projected/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-api-access-447l8\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.829177 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.829036 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.829588 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.829553 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78194fb5-9251-42bd-bba8-2849f2223f5f-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.829802 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.829781 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.829939 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.829916 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.831463 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.831444 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78194fb5-9251-42bd-bba8-2849f2223f5f-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.840616 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.840587 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:56.843532 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.843512 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-447l8\" (UniqueName: \"kubernetes.io/projected/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-api-access-447l8\") pod \"isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.935744 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.935713 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:56.957254 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.957226 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp"] Sep 18 19:25:56.961016 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.961000 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:56.961079 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.961047 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:25:56.963222 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.963198 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Sep 18 19:25:56.963341 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.963291 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Sep 18 19:25:56.972826 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:56.972799 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp"] Sep 18 19:25:57.004656 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.004587 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:57.030535 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.030225 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.030535 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.030294 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.030535 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.030349 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.030535 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.030390 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkfzr\" (UniqueName: \"kubernetes.io/projected/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-api-access-bkfzr\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.030535 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.030441 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-proxy-tls\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.091163 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.091090 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:57.131394 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.131350 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkfzr\" (UniqueName: \"kubernetes.io/projected/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-api-access-bkfzr\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.131599 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.131422 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-proxy-tls\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.131599 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.131473 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.131599 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.131520 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.131599 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.131567 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.132173 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:25:57.131965 2554 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-predictor-serving-cert: secret "isvc-xgboost-graph-predictor-serving-cert" not found Sep 18 19:25:57.132173 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:25:57.132038 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-proxy-tls podName:2bacc262-5ee4-421a-ab8c-ff4c994c09e6 nodeName:}" failed. No retries permitted until 2026-09-18 19:25:57.632017824 +0000 UTC m=+703.536365573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-proxy-tls") pod "isvc-xgboost-graph-predictor-d7b8766ff-zndlp" (UID: "2bacc262-5ee4-421a-ab8c-ff4c994c09e6") : secret "isvc-xgboost-graph-predictor-serving-cert" not found Sep 18 19:25:57.132331 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.132280 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.132376 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.132351 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.132487 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.132466 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.144308 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.144287 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkfzr\" (UniqueName: \"kubernetes.io/projected/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-api-access-bkfzr\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.239391 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.239366 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz"] Sep 18 19:25:57.241382 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:25:57.241355 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78194fb5_9251_42bd_bba8_2849f2223f5f.slice/crio-180f6a5bd9c0276abdd1a3e8fa44c731328e85e4bd66942325c94790926c1cc1 WatchSource:0}: Error finding container 180f6a5bd9c0276abdd1a3e8fa44c731328e85e4bd66942325c94790926c1cc1: Status 404 returned error can't find the container with id 180f6a5bd9c0276abdd1a3e8fa44c731328e85e4bd66942325c94790926c1cc1 Sep 18 19:25:57.302239 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.302193 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:57.409196 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.409154 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:25:57.636782 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.636693 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-proxy-tls\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.639089 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.639061 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-proxy-tls\") pod \"isvc-xgboost-graph-predictor-d7b8766ff-zndlp\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:57.885005 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:57.884967 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:25:58.039054 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:58.039017 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" event={"ID":"78194fb5-9251-42bd-bba8-2849f2223f5f","Type":"ContainerStarted","Data":"100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c"} Sep 18 19:25:58.039221 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:58.039083 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" event={"ID":"78194fb5-9251-42bd-bba8-2849f2223f5f","Type":"ContainerStarted","Data":"180f6a5bd9c0276abdd1a3e8fa44c731328e85e4bd66942325c94790926c1cc1"} Sep 18 19:25:58.158056 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:58.157885 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp"] Sep 18 19:25:58.163125 ip-10-0-129-212 kubenswrapper[2554]: W0918 19:25:58.160628 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bacc262_5ee4_421a_ab8c_ff4c994c09e6.slice/crio-6b1546a06e85e42078ee069a452f8497cc606b1c13708340adee672c4fda104b WatchSource:0}: Error finding container 6b1546a06e85e42078ee069a452f8497cc606b1c13708340adee672c4fda104b: Status 404 returned error can't find the container with id 6b1546a06e85e42078ee069a452f8497cc606b1c13708340adee672c4fda104b Sep 18 19:25:59.044873 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:59.044833 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" event={"ID":"2bacc262-5ee4-421a-ab8c-ff4c994c09e6","Type":"ContainerStarted","Data":"efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1"} Sep 18 19:25:59.045330 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:25:59.044893 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" event={"ID":"2bacc262-5ee4-421a-ab8c-ff4c994c09e6","Type":"ContainerStarted","Data":"6b1546a06e85e42078ee069a452f8497cc606b1c13708340adee672c4fda104b"} Sep 18 19:26:02.059465 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:02.059427 2554 generic.go:356] "Generic (PLEG): container finished" podID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerID="100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c" exitCode=0 Sep 18 19:26:02.059859 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:02.059482 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" event={"ID":"78194fb5-9251-42bd-bba8-2849f2223f5f","Type":"ContainerDied","Data":"100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c"} Sep 18 19:26:03.069543 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:03.069504 2554 generic.go:356] "Generic (PLEG): container finished" podID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerID="efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1" exitCode=0 Sep 18 19:26:03.070008 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:03.069671 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" event={"ID":"2bacc262-5ee4-421a-ab8c-ff4c994c09e6","Type":"ContainerDied","Data":"efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1"} Sep 18 19:26:27.199922 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:27.199885 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" event={"ID":"78194fb5-9251-42bd-bba8-2849f2223f5f","Type":"ContainerStarted","Data":"48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6"} Sep 18 19:26:28.217097 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:28.215782 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" event={"ID":"2bacc262-5ee4-421a-ab8c-ff4c994c09e6","Type":"ContainerStarted","Data":"9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356"} Sep 18 19:26:30.227674 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:30.227615 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" event={"ID":"78194fb5-9251-42bd-bba8-2849f2223f5f","Type":"ContainerStarted","Data":"ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b"} Sep 18 19:26:30.228127 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:30.227708 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:26:30.228127 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:30.227744 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:26:30.229235 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:30.229201 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Sep 18 19:26:30.229967 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:30.229944 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" event={"ID":"2bacc262-5ee4-421a-ab8c-ff4c994c09e6","Type":"ContainerStarted","Data":"eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c"} Sep 18 19:26:30.230070 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:30.230054 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:26:30.230118 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:30.230102 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:26:30.231137 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:30.231117 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Sep 18 19:26:30.271199 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:30.271148 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podStartSLOduration=2.8676442939999998 podStartE2EDuration="34.271132461s" totalImagesPullingTime="27.62945893s" totalInitContainerRuntime="3.774029237s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:25:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:26:30.270575248 +0000 UTC m=+736.174923005" watchObservedRunningTime="2026-09-18 19:26:30.271132461 +0000 UTC m=+736.175480217" Sep 18 19:26:30.330571 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:30.330515 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" podStartSLOduration=3.825374778 podStartE2EDuration="34.330497922s" totalImagesPullingTime="26.615789101s" totalInitContainerRuntime="3.889334043s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:25:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:26:30.330237844 +0000 UTC m=+736.234585602" watchObservedRunningTime="2026-09-18 19:26:30.330497922 +0000 UTC m=+736.234845679" Sep 18 19:26:31.234619 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:31.234575 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Sep 18 19:26:31.234619 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:31.234602 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Sep 18 19:26:36.239499 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:36.239470 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:26:36.239986 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:36.239966 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:26:36.240051 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:36.240026 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Sep 18 19:26:36.240423 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:36.240402 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Sep 18 19:26:46.240702 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:46.240661 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Sep 18 19:26:46.241085 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:46.240665 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Sep 18 19:26:52.202900 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:52.202867 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:26:53.303251 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:53.303216 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:26:56.240119 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:56.240077 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Sep 18 19:26:56.240693 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:26:56.240517 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Sep 18 19:27:06.240879 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:06.240837 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Sep 18 19:27:06.241314 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:06.240837 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Sep 18 19:27:09.636325 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:09.636288 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:27:16.240333 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:16.240295 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Sep 18 19:27:16.240686 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:16.240425 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Sep 18 19:27:20.146669 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:20.146607 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:27:20.577387 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:20.577302 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:27:26.240484 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:26.240445 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Sep 18 19:27:26.241002 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:26.240448 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Sep 18 19:27:31.179165 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:31.179082 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:27:31.316872 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:31.316823 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:27:31.418105 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:31.418074 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:27:31.501476 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:31.501386 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:27:31.756480 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:31.756397 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:27:31.854543 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:31.854514 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:27:36.240480 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:36.240435 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Sep 18 19:27:36.240888 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:36.240812 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:27:46.240826 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:27:46.240751 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:28:03.386535 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:03.386498 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:10.094626 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:10.094592 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:10.369022 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:10.368943 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:21.514950 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:21.514914 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:21.642749 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:21.642718 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz"] Sep 18 19:28:21.643095 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:21.643054 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" containerID="cri-o://48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6" gracePeriod=30 Sep 18 19:28:21.643337 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:21.643116 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kube-rbac-proxy" containerID="cri-o://ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b" gracePeriod=30 Sep 18 19:28:21.647086 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:21.647058 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:21.685602 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:21.685568 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp"] Sep 18 19:28:21.685828 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:21.685797 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" containerID="cri-o://9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356" gracePeriod=30 Sep 18 19:28:21.685977 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:21.685842 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kube-rbac-proxy" containerID="cri-o://eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c" gracePeriod=30 Sep 18 19:28:21.748047 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:21.748014 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:21.844082 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:21.844049 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:21.919792 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:21.919749 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:22.389490 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:22.389441 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:22.684572 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:22.684491 2554 generic.go:356] "Generic (PLEG): container finished" podID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerID="eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c" exitCode=2 Sep 18 19:28:22.684572 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:22.684565 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" event={"ID":"2bacc262-5ee4-421a-ab8c-ff4c994c09e6","Type":"ContainerDied","Data":"eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c"} Sep 18 19:28:22.686699 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:22.686679 2554 generic.go:356] "Generic (PLEG): container finished" podID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerID="ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b" exitCode=2 Sep 18 19:28:22.686804 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:22.686732 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" event={"ID":"78194fb5-9251-42bd-bba8-2849f2223f5f","Type":"ContainerDied","Data":"ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b"} Sep 18 19:28:22.706713 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:22.706693 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:25.929047 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:25.929024 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:28:26.029502 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.029432 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-rbac-proxy-sar-config\") pod \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " Sep 18 19:28:26.029502 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.029482 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-cabundle-cert\") pod \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " Sep 18 19:28:26.029742 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.029606 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kserve-provision-location\") pod \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " Sep 18 19:28:26.029742 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.029708 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-proxy-tls\") pod \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " Sep 18 19:28:26.029861 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.029764 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkfzr\" (UniqueName: \"kubernetes.io/projected/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-api-access-bkfzr\") pod \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\" (UID: \"2bacc262-5ee4-421a-ab8c-ff4c994c09e6\") " Sep 18 19:28:26.029916 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.029864 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "2bacc262-5ee4-421a-ab8c-ff4c994c09e6" (UID: "2bacc262-5ee4-421a-ab8c-ff4c994c09e6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:28:26.029959 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.029901 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2bacc262-5ee4-421a-ab8c-ff4c994c09e6" (UID: "2bacc262-5ee4-421a-ab8c-ff4c994c09e6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:28:26.029959 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.029910 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2bacc262-5ee4-421a-ab8c-ff4c994c09e6" (UID: "2bacc262-5ee4-421a-ab8c-ff4c994c09e6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 19:28:26.030026 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.030017 2554 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kserve-provision-location\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:28:26.030063 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.030032 2554 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:28:26.030063 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.030045 2554 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-cabundle-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:28:26.031849 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.031819 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2bacc262-5ee4-421a-ab8c-ff4c994c09e6" (UID: "2bacc262-5ee4-421a-ab8c-ff4c994c09e6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:28:26.031948 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.031848 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-api-access-bkfzr" (OuterVolumeSpecName: "kube-api-access-bkfzr") pod "2bacc262-5ee4-421a-ab8c-ff4c994c09e6" (UID: "2bacc262-5ee4-421a-ab8c-ff4c994c09e6"). InnerVolumeSpecName "kube-api-access-bkfzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 19:28:26.130975 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.130940 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkfzr\" (UniqueName: \"kubernetes.io/projected/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-kube-api-access-bkfzr\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:28:26.130975 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.130968 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2bacc262-5ee4-421a-ab8c-ff4c994c09e6-proxy-tls\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:28:26.235555 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.235517 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.23:8643/healthz\": dial tcp 10.133.0.23:8643: connect: connection refused" Sep 18 19:28:26.241117 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.241094 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Sep 18 19:28:26.682387 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.682361 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:28:26.704950 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.704918 2554 generic.go:356] "Generic (PLEG): container finished" podID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerID="9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356" exitCode=0 Sep 18 19:28:26.705089 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.704968 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" event={"ID":"2bacc262-5ee4-421a-ab8c-ff4c994c09e6","Type":"ContainerDied","Data":"9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356"} Sep 18 19:28:26.705089 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.705014 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" Sep 18 19:28:26.705089 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.705056 2554 scope.go:117] "RemoveContainer" containerID="eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c" Sep 18 19:28:26.705260 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.705040 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp" event={"ID":"2bacc262-5ee4-421a-ab8c-ff4c994c09e6","Type":"ContainerDied","Data":"6b1546a06e85e42078ee069a452f8497cc606b1c13708340adee672c4fda104b"} Sep 18 19:28:26.708129 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.708008 2554 generic.go:356] "Generic (PLEG): container finished" podID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerID="48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6" exitCode=0 Sep 18 19:28:26.708129 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.708047 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" event={"ID":"78194fb5-9251-42bd-bba8-2849f2223f5f","Type":"ContainerDied","Data":"48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6"} Sep 18 19:28:26.708129 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.708088 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" Sep 18 19:28:26.708129 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.708094 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz" event={"ID":"78194fb5-9251-42bd-bba8-2849f2223f5f","Type":"ContainerDied","Data":"180f6a5bd9c0276abdd1a3e8fa44c731328e85e4bd66942325c94790926c1cc1"} Sep 18 19:28:26.714526 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.714507 2554 scope.go:117] "RemoveContainer" containerID="9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356" Sep 18 19:28:26.722372 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.722355 2554 scope.go:117] "RemoveContainer" containerID="efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1" Sep 18 19:28:26.725760 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.725741 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp"] Sep 18 19:28:26.730666 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.730650 2554 scope.go:117] "RemoveContainer" containerID="eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c" Sep 18 19:28:26.730727 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.730678 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-d7b8766ff-zndlp"] Sep 18 19:28:26.730966 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:28:26.730935 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c\": container with ID starting with eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c not found: ID does not exist" containerID="eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c" Sep 18 19:28:26.731059 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.730966 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c"} err="failed to get container status \"eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c\": rpc error: code = NotFound desc = could not find container \"eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c\": container with ID starting with eb0c8f9fb2ab1b008ab31a76cc63edd5ba15ca95f6449b0b34aae6cd6e57221c not found: ID does not exist" Sep 18 19:28:26.731059 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.730984 2554 scope.go:117] "RemoveContainer" containerID="9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356" Sep 18 19:28:26.731272 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:28:26.731254 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356\": container with ID starting with 9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356 not found: ID does not exist" containerID="9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356" Sep 18 19:28:26.731321 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.731278 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356"} err="failed to get container status \"9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356\": rpc error: code = NotFound desc = could not find container \"9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356\": container with ID starting with 9f3f6025d1fb0805abe346272c7c3c3c4a41257c6cef05eab98014e9c0f8b356 not found: ID does not exist" Sep 18 19:28:26.731321 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.731293 2554 scope.go:117] "RemoveContainer" containerID="efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1" Sep 18 19:28:26.731508 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:28:26.731493 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1\": container with ID starting with efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1 not found: ID does not exist" containerID="efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1" Sep 18 19:28:26.731549 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.731512 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1"} err="failed to get container status \"efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1\": rpc error: code = NotFound desc = could not find container \"efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1\": container with ID starting with efed565083b3bddcb91c53b930defa419ea2e7f576a912fca7c1cf7049cd41a1 not found: ID does not exist" Sep 18 19:28:26.731549 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.731526 2554 scope.go:117] "RemoveContainer" containerID="ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b" Sep 18 19:28:26.736570 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.736551 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78194fb5-9251-42bd-bba8-2849f2223f5f-proxy-tls\") pod \"78194fb5-9251-42bd-bba8-2849f2223f5f\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " Sep 18 19:28:26.736680 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.736630 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-rbac-proxy-sar-config\") pod \"78194fb5-9251-42bd-bba8-2849f2223f5f\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " Sep 18 19:28:26.736761 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.736689 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-cabundle-cert\") pod \"78194fb5-9251-42bd-bba8-2849f2223f5f\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " Sep 18 19:28:26.736761 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.736727 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-447l8\" (UniqueName: \"kubernetes.io/projected/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-api-access-447l8\") pod \"78194fb5-9251-42bd-bba8-2849f2223f5f\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " Sep 18 19:28:26.736873 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.736760 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78194fb5-9251-42bd-bba8-2849f2223f5f-kserve-provision-location\") pod \"78194fb5-9251-42bd-bba8-2849f2223f5f\" (UID: \"78194fb5-9251-42bd-bba8-2849f2223f5f\") " Sep 18 19:28:26.737027 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.736998 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "78194fb5-9251-42bd-bba8-2849f2223f5f" (UID: "78194fb5-9251-42bd-bba8-2849f2223f5f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:28:26.737161 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.737054 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "78194fb5-9251-42bd-bba8-2849f2223f5f" (UID: "78194fb5-9251-42bd-bba8-2849f2223f5f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 19:28:26.737224 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.737196 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78194fb5-9251-42bd-bba8-2849f2223f5f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "78194fb5-9251-42bd-bba8-2849f2223f5f" (UID: "78194fb5-9251-42bd-bba8-2849f2223f5f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 19:28:26.738699 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.738577 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78194fb5-9251-42bd-bba8-2849f2223f5f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "78194fb5-9251-42bd-bba8-2849f2223f5f" (UID: "78194fb5-9251-42bd-bba8-2849f2223f5f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 19:28:26.738768 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.738705 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-api-access-447l8" (OuterVolumeSpecName: "kube-api-access-447l8") pod "78194fb5-9251-42bd-bba8-2849f2223f5f" (UID: "78194fb5-9251-42bd-bba8-2849f2223f5f"). InnerVolumeSpecName "kube-api-access-447l8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 19:28:26.738768 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.738748 2554 scope.go:117] "RemoveContainer" containerID="48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6" Sep 18 19:28:26.745519 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.745503 2554 scope.go:117] "RemoveContainer" containerID="100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c" Sep 18 19:28:26.752247 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.752228 2554 scope.go:117] "RemoveContainer" containerID="ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b" Sep 18 19:28:26.752507 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:28:26.752489 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b\": container with ID starting with ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b not found: ID does not exist" containerID="ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b" Sep 18 19:28:26.752562 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.752520 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b"} err="failed to get container status \"ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b\": rpc error: code = NotFound desc = could not find container \"ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b\": container with ID starting with ee4f91b629a1e3ef3a333d480d68933d8ba2da35d55cab62d08978000ef6fe2b not found: ID does not exist" Sep 18 19:28:26.752562 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.752535 2554 scope.go:117] "RemoveContainer" containerID="48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6" Sep 18 19:28:26.752801 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:28:26.752772 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6\": container with ID starting with 48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6 not found: ID does not exist" containerID="48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6" Sep 18 19:28:26.752840 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.752810 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6"} err="failed to get container status \"48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6\": rpc error: code = NotFound desc = could not find container \"48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6\": container with ID starting with 48cdd2b997377ee5db7777db53c0456b451107f5147c6aa7c0a02aef687a60f6 not found: ID does not exist" Sep 18 19:28:26.752840 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.752830 2554 scope.go:117] "RemoveContainer" containerID="100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c" Sep 18 19:28:26.753080 ip-10-0-129-212 kubenswrapper[2554]: E0918 19:28:26.753065 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c\": container with ID starting with 100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c not found: ID does not exist" containerID="100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c" Sep 18 19:28:26.753118 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.753084 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c"} err="failed to get container status \"100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c\": rpc error: code = NotFound desc = could not find container \"100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c\": container with ID starting with 100ce5d1d0eb9f01c89ea65affaac437b3369dfe7419438283433b06fc650f0c not found: ID does not exist" Sep 18 19:28:26.837959 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.837883 2554 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:28:26.837959 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.837910 2554 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78194fb5-9251-42bd-bba8-2849f2223f5f-cabundle-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:28:26.837959 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.837923 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-447l8\" (UniqueName: \"kubernetes.io/projected/78194fb5-9251-42bd-bba8-2849f2223f5f-kube-api-access-447l8\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:28:26.837959 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.837931 2554 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78194fb5-9251-42bd-bba8-2849f2223f5f-kserve-provision-location\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:28:26.837959 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:26.837941 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78194fb5-9251-42bd-bba8-2849f2223f5f-proxy-tls\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 19:28:27.038022 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:27.037991 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz"] Sep 18 19:28:27.041351 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:27.041327 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6589c5859b-gcdzz"] Sep 18 19:28:28.276740 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:28.276701 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" path="/var/lib/kubelet/pods/2bacc262-5ee4-421a-ab8c-ff4c994c09e6/volumes" Sep 18 19:28:28.277380 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:28.277359 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" path="/var/lib/kubelet/pods/78194fb5-9251-42bd-bba8-2849f2223f5f/volumes" Sep 18 19:28:31.461085 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:31.461036 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:31.886500 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:31.886463 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:28:52.887057 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:28:52.887018 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:29:01.947570 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:29:01.947485 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:29:02.012987 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:29:02.012953 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:29:16.169122 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:29:16.169097 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:29:16.170839 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:29:16.170816 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:29:21.715516 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:29:21.715482 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:29:21.981801 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:29:21.981716 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:30:36.389424 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:30:36.389339 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:31:35.315986 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:31:35.315950 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:31:35.398812 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:31:35.398764 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:32:57.687138 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:32:57.687102 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:34:16.194406 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:34:16.194377 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:34:16.197446 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:34:16.197417 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:36:29.540669 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:36:29.540572 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:36:46.548536 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:36:46.548504 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:36:46.690983 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:36:46.690948 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:36:46.772753 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:36:46.772720 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:36:46.853688 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:36:46.853594 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:36:46.957732 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:36:46.957691 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:37.282973 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:37.282939 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:37.405996 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:37.405960 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:37.500931 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:37.500897 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:37.583740 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:37.583660 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:37.672655 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:37.672611 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:37.762786 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:37.762743 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:46.814043 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:46.814000 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:46.976792 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:46.976754 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:57.319914 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:57.319878 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:57.421484 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:57.421451 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:57.511091 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:57.511060 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:57.573066 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:57.572981 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:57.650047 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:57.649977 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:37:57.894629 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:37:57.894598 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:38:06.214491 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:38:06.214461 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:38:47.525318 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:38:47.525280 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:38:47.808360 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:38:47.808267 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:38:57.577611 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:38:57.577571 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:38:57.824166 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:38:57.824127 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:38:58.480779 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:38:58.480743 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:38:58.597051 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:38:58.597020 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:38:58.704945 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:38:58.704908 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:38:58.797563 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:38:58.797487 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:38:58.872840 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:38:58.872804 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:38:58.940191 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:38:58.940153 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:39:16.219718 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:39:16.219669 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:39:16.223706 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:39:16.223685 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:39:58.743774 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:39:58.743743 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:39:58.881429 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:39:58.881397 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:43:48.352014 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:43:48.351981 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:44:16.244725 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:44:16.244701 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:44:16.249209 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:44:16.249182 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:45:10.880723 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:45:10.880688 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:47:12.681907 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:47:12.681827 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:47:13.029964 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:47:13.029899 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:47:13.156261 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:47:13.156233 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:47:13.277896 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:47:13.277861 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:47:13.415451 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:47:13.415425 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:47:13.582244 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:47:13.582212 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:47:13.724489 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:47:13.724389 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:12.997999 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:12.997948 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:13.383172 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:13.383144 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:13.762987 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:13.762884 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:13.960859 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:13.960829 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:14.103131 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:14.103026 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:14.221021 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:14.220987 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:14.629063 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:14.629028 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:28.107514 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:28.107418 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:28.354892 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:28.354856 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:28.535210 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:28.535171 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:28.711282 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:28.711242 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:28.814247 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:28.814149 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:48:29.142532 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:48:29.142498 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:49:14.111591 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:49:14.111557 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:49:15.449013 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:49:15.448979 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:49:16.270239 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:49:16.270210 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:49:16.276173 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:49:16.276150 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:49:28.333370 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:49:28.333334 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:49:28.654481 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:49:28.654449 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:50:23.798418 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:50:23.798337 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:51:22.234214 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:51:22.234182 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:52:04.423966 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:52:04.423935 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:54:16.299832 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:54:16.299800 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:54:16.302313 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:54:16.301943 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:56:05.403469 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:56:05.403386 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:57:43.532329 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:57:43.532250 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:57:43.621500 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:57:43.621464 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:57:43.720946 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:57:43.720915 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 19:59:16.324996 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:59:16.324887 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 19:59:16.328245 ip-10-0-129-212 kubenswrapper[2554]: I0918 19:59:16.327147 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 20:00:25.591996 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:00:25.591962 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:04:16.350202 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:04:16.350082 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 20:04:16.353166 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:04:16.353151 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 20:05:19.754487 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:19.754396 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:05:34.242633 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:34.242591 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:05:34.397567 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:34.397528 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:05:34.482253 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:34.482194 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:05:35.453657 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.453601 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bwg64/must-gather-2zrvf"] Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.453933 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="storage-initializer" Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.453946 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="storage-initializer" Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.453958 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.453964 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.453979 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kube-rbac-proxy" Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.453984 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kube-rbac-proxy" Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.453994 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="storage-initializer" Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.453999 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="storage-initializer" Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.454005 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kube-rbac-proxy" Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.454011 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kube-rbac-proxy" Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.454017 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" Sep 18 20:05:35.454042 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.454022 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" Sep 18 20:05:35.454528 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.454066 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kserve-container" Sep 18 20:05:35.454528 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.454074 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kube-rbac-proxy" Sep 18 20:05:35.454528 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.454082 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bacc262-5ee4-421a-ab8c-ff4c994c09e6" containerName="kserve-container" Sep 18 20:05:35.454528 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.454088 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="78194fb5-9251-42bd-bba8-2849f2223f5f" containerName="kube-rbac-proxy" Sep 18 20:05:35.457275 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.457254 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:05:35.463108 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.463081 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bwg64\"/\"default-dockercfg-dldsq\"" Sep 18 20:05:35.463108 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.463095 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bwg64\"/\"kube-root-ca.crt\"" Sep 18 20:05:35.463909 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.463809 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bwg64\"/\"openshift-service-ca.crt\"" Sep 18 20:05:35.468670 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.468621 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bwg64/must-gather-2zrvf"] Sep 18 20:05:35.495674 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.495621 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:05:35.542883 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.542845 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-must-gather-output\") pod \"must-gather-2zrvf\" (UID: \"78807d96-3f53-4ed3-ac7f-b76f0b4862c8\") " pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:05:35.543017 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.542894 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt492\" (UniqueName: \"kubernetes.io/projected/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-kube-api-access-gt492\") pod \"must-gather-2zrvf\" (UID: \"78807d96-3f53-4ed3-ac7f-b76f0b4862c8\") " pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:05:35.644030 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.643995 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-must-gather-output\") pod \"must-gather-2zrvf\" (UID: \"78807d96-3f53-4ed3-ac7f-b76f0b4862c8\") " pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:05:35.644142 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.644045 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt492\" (UniqueName: \"kubernetes.io/projected/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-kube-api-access-gt492\") pod \"must-gather-2zrvf\" (UID: \"78807d96-3f53-4ed3-ac7f-b76f0b4862c8\") " pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:05:35.644342 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.644321 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-must-gather-output\") pod \"must-gather-2zrvf\" (UID: \"78807d96-3f53-4ed3-ac7f-b76f0b4862c8\") " pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:05:35.652984 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.652948 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt492\" (UniqueName: \"kubernetes.io/projected/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-kube-api-access-gt492\") pod \"must-gather-2zrvf\" (UID: \"78807d96-3f53-4ed3-ac7f-b76f0b4862c8\") " pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:05:35.685277 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.685222 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:05:35.781233 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:35.781145 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:05:36.069333 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:36.069248 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bwg64/must-gather-2zrvf"] Sep 18 20:05:36.073364 ip-10-0-129-212 kubenswrapper[2554]: W0918 20:05:36.073328 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78807d96_3f53_4ed3_ac7f_b76f0b4862c8.slice/crio-a35807cd8310c7b168b71291ef8cbf44d0b64c57443561cedf759f7d7b03b341 WatchSource:0}: Error finding container a35807cd8310c7b168b71291ef8cbf44d0b64c57443561cedf759f7d7b03b341: Status 404 returned error can't find the container with id a35807cd8310c7b168b71291ef8cbf44d0b64c57443561cedf759f7d7b03b341 Sep 18 20:05:36.075275 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:36.075252 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 20:05:36.293426 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:36.293392 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwg64/must-gather-2zrvf" event={"ID":"78807d96-3f53-4ed3-ac7f-b76f0b4862c8","Type":"ContainerStarted","Data":"a35807cd8310c7b168b71291ef8cbf44d0b64c57443561cedf759f7d7b03b341"} Sep 18 20:05:43.330145 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:43.330107 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwg64/must-gather-2zrvf" event={"ID":"78807d96-3f53-4ed3-ac7f-b76f0b4862c8","Type":"ContainerStarted","Data":"e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b"} Sep 18 20:05:43.330543 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:43.330159 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwg64/must-gather-2zrvf" event={"ID":"78807d96-3f53-4ed3-ac7f-b76f0b4862c8","Type":"ContainerStarted","Data":"b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be"} Sep 18 20:05:52.025979 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:05:52.025936 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:06:05.419997 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:05.419956 2554 generic.go:356] "Generic (PLEG): container finished" podID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" containerID="b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be" exitCode=0 Sep 18 20:06:05.420411 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:05.420024 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwg64/must-gather-2zrvf" event={"ID":"78807d96-3f53-4ed3-ac7f-b76f0b4862c8","Type":"ContainerDied","Data":"b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be"} Sep 18 20:06:05.420411 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:05.420339 2554 scope.go:117] "RemoveContainer" containerID="b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be" Sep 18 20:06:05.653918 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:05.653875 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bwg64_must-gather-2zrvf_78807d96-3f53-4ed3-ac7f-b76f0b4862c8/gather/0.log" Sep 18 20:06:06.140417 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:06.140368 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:06:06.387709 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:06.387673 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:06:06.554619 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:06.554515 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:06:09.247171 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:09.247137 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nphvd_d7623d04-8395-4bc7-b628-d82e5e22b847/global-pull-secret-syncer/0.log" Sep 18 20:06:09.355745 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:09.355711 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" Sep 18 20:06:09.466236 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:09.466200 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-212.ec2.internal_4c6c0e083d554061009b46141d9770d5/haproxy/0.log" Sep 18 20:06:11.105326 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.105289 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-bwg64/must-gather-2zrvf"] Sep 18 20:06:11.105779 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.105481 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-bwg64/must-gather-2zrvf" podUID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" containerName="copy" containerID="cri-o://e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b" gracePeriod=2 Sep 18 20:06:11.111246 ip-10-0-129-212 kubenswrapper[2554]: E0918 20:06:11.111201 2554 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"78807d96-3f53-4ed3-ac7f-b76f0b4862c8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"openshift-must-gather-bwg64\"/\"must-gather-2zrvf\": pods \"must-gather-2zrvf\" not found" pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:06:11.111834 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.111808 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-bwg64/must-gather-2zrvf"] Sep 18 20:06:11.339393 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.339366 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bwg64_must-gather-2zrvf_78807d96-3f53-4ed3-ac7f-b76f0b4862c8/copy/0.log" Sep 18 20:06:11.341916 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.341896 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:06:11.343763 ip-10-0-129-212 kubenswrapper[2554]: E0918 20:06:11.343737 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-2zrvf\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bwg64\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:06:11.445161 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.445130 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bwg64_must-gather-2zrvf_78807d96-3f53-4ed3-ac7f-b76f0b4862c8/copy/0.log" Sep 18 20:06:11.445565 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.445539 2554 generic.go:356] "Generic (PLEG): container finished" podID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" containerID="e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b" exitCode=143 Sep 18 20:06:11.445664 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.445595 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:06:11.445664 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.445612 2554 scope.go:117] "RemoveContainer" containerID="e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b" Sep 18 20:06:11.447887 ip-10-0-129-212 kubenswrapper[2554]: E0918 20:06:11.447850 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-2zrvf\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bwg64\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:06:11.453460 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.453439 2554 scope.go:117] "RemoveContainer" containerID="b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be" Sep 18 20:06:11.461727 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.461701 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-must-gather-output\") pod \"78807d96-3f53-4ed3-ac7f-b76f0b4862c8\" (UID: \"78807d96-3f53-4ed3-ac7f-b76f0b4862c8\") " Sep 18 20:06:11.461852 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.461760 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gt492\" (UniqueName: \"kubernetes.io/projected/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-kube-api-access-gt492\") pod \"78807d96-3f53-4ed3-ac7f-b76f0b4862c8\" (UID: \"78807d96-3f53-4ed3-ac7f-b76f0b4862c8\") " Sep 18 20:06:11.463196 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.463166 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "78807d96-3f53-4ed3-ac7f-b76f0b4862c8" (UID: "78807d96-3f53-4ed3-ac7f-b76f0b4862c8"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 20:06:11.463923 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.463898 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-kube-api-access-gt492" (OuterVolumeSpecName: "kube-api-access-gt492") pod "78807d96-3f53-4ed3-ac7f-b76f0b4862c8" (UID: "78807d96-3f53-4ed3-ac7f-b76f0b4862c8"). InnerVolumeSpecName "kube-api-access-gt492". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 20:06:11.466319 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.466301 2554 scope.go:117] "RemoveContainer" containerID="e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b" Sep 18 20:06:11.466633 ip-10-0-129-212 kubenswrapper[2554]: E0918 20:06:11.466613 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b\": container with ID starting with e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b not found: ID does not exist" containerID="e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b" Sep 18 20:06:11.466717 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.466656 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b"} err="failed to get container status \"e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b\": rpc error: code = NotFound desc = could not find container \"e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b\": container with ID starting with e28bb2f13090f92fcd43edc71d64d93d89b58e9a4e8efc3c3bd7d8bc2843f19b not found: ID does not exist" Sep 18 20:06:11.466717 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.466676 2554 scope.go:117] "RemoveContainer" containerID="b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be" Sep 18 20:06:11.466909 ip-10-0-129-212 kubenswrapper[2554]: E0918 20:06:11.466895 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be\": container with ID starting with b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be not found: ID does not exist" containerID="b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be" Sep 18 20:06:11.466948 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.466913 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be"} err="failed to get container status \"b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be\": rpc error: code = NotFound desc = could not find container \"b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be\": container with ID starting with b6663390dfb998965c38c1260471f94cad0e1bd1c2aa5524e9629508923ab9be not found: ID does not exist" Sep 18 20:06:11.562579 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.562535 2554 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-must-gather-output\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 20:06:11.562579 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:11.562572 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt492\" (UniqueName: \"kubernetes.io/projected/78807d96-3f53-4ed3-ac7f-b76f0b4862c8-kube-api-access-gt492\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Sep 18 20:06:11.756517 ip-10-0-129-212 kubenswrapper[2554]: E0918 20:06:11.756486 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-2zrvf\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bwg64\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" pod="openshift-must-gather-bwg64/must-gather-2zrvf" Sep 18 20:06:12.155396 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:12.155357 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:06:12.234361 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:12.234320 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:06:12.277041 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:12.277012 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" path="/var/lib/kubelet/pods/78807d96-3f53-4ed3-ac7f-b76f0b4862c8/volumes" Sep 18 20:06:13.717606 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:13.717564 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kfxm_d0e6fa6e-811a-4d85-a190-a27c5563dfd9/konnectivity-agent/0.log" limit=1024 Sep 18 20:06:14.027601 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.027560 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-t7kxg_6da8bc41-9fa5-4802-8f1f-08e96fed5f60/monitoring-plugin/0.log" Sep 18 20:06:14.302087 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.302006 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fj5wk_67873421-8959-4e5d-a3b0-e2b650ff1292/node-exporter/0.log" Sep 18 20:06:14.353079 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.353029 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fj5wk_67873421-8959-4e5d-a3b0-e2b650ff1292/kube-rbac-proxy/0.log" Sep 18 20:06:14.384045 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.384016 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fj5wk_67873421-8959-4e5d-a3b0-e2b650ff1292/init-textfile/0.log" Sep 18 20:06:14.437515 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.437458 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-754fd_976f4b92-c892-4be8-8cd8-db5d1f03bcc2/kube-rbac-proxy-main/0.log" Sep 18 20:06:14.474405 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.474365 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-754fd_976f4b92-c892-4be8-8cd8-db5d1f03bcc2/kube-rbac-proxy-self/0.log" Sep 18 20:06:14.520810 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.520772 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-754fd_976f4b92-c892-4be8-8cd8-db5d1f03bcc2/openshift-state-metrics/0.log" Sep 18 20:06:14.595086 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.595059 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c4b39b77-a033-49f0-a606-59152138a433/prometheus/0.log" Sep 18 20:06:14.618422 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.618387 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c4b39b77-a033-49f0-a606-59152138a433/config-reloader/0.log" Sep 18 20:06:14.660869 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.660828 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c4b39b77-a033-49f0-a606-59152138a433/thanos-sidecar/0.log" Sep 18 20:06:14.684099 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.684068 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c4b39b77-a033-49f0-a606-59152138a433/kube-rbac-proxy-web/0.log" Sep 18 20:06:14.709304 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.709274 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c4b39b77-a033-49f0-a606-59152138a433/kube-rbac-proxy/0.log" Sep 18 20:06:14.738186 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.738157 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c4b39b77-a033-49f0-a606-59152138a433/kube-rbac-proxy-thanos/0.log" Sep 18 20:06:14.766584 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.766553 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c4b39b77-a033-49f0-a606-59152138a433/init-config-reloader/0.log" Sep 18 20:06:14.813183 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.813152 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-dv7rb_331a3f0f-55b3-4e1d-83ec-e6fea064ffd5/prometheus-operator/0.log" Sep 18 20:06:14.831770 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.831740 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-dv7rb_331a3f0f-55b3-4e1d-83ec-e6fea064ffd5/kube-rbac-proxy/0.log" Sep 18 20:06:14.862353 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:14.862262 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-vkktt_0907315e-5c19-424a-b15b-6bfce3a15da7/prometheus-operator-admission-webhook/0.log" Sep 18 20:06:15.039816 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:15.039780 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68665cdd46-v65g5_3ae18d24-a8fa-4605-bd57-0fb210b92845/thanos-query/0.log" Sep 18 20:06:15.075137 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:15.075108 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68665cdd46-v65g5_3ae18d24-a8fa-4605-bd57-0fb210b92845/kube-rbac-proxy-web/0.log" Sep 18 20:06:15.108721 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:15.108693 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68665cdd46-v65g5_3ae18d24-a8fa-4605-bd57-0fb210b92845/kube-rbac-proxy/0.log" Sep 18 20:06:15.138587 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:15.138560 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68665cdd46-v65g5_3ae18d24-a8fa-4605-bd57-0fb210b92845/prom-label-proxy/0.log" Sep 18 20:06:15.173024 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:15.172986 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68665cdd46-v65g5_3ae18d24-a8fa-4605-bd57-0fb210b92845/kube-rbac-proxy-rules/0.log" Sep 18 20:06:15.202602 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:15.202570 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68665cdd46-v65g5_3ae18d24-a8fa-4605-bd57-0fb210b92845/kube-rbac-proxy-metrics/0.log" Sep 18 20:06:16.952891 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.952848 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t"] Sep 18 20:06:16.953319 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.953187 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" containerName="copy" Sep 18 20:06:16.953319 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.953200 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" containerName="copy" Sep 18 20:06:16.953319 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.953218 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" containerName="gather" Sep 18 20:06:16.953319 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.953223 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" containerName="gather" Sep 18 20:06:16.953319 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.953285 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" containerName="gather" Sep 18 20:06:16.953319 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.953295 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="78807d96-3f53-4ed3-ac7f-b76f0b4862c8" containerName="copy" Sep 18 20:06:16.956083 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.956065 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:16.961123 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.961100 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7jg6d\"/\"kube-root-ca.crt\"" Sep 18 20:06:16.964421 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.964399 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7jg6d\"/\"openshift-service-ca.crt\"" Sep 18 20:06:16.969991 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.969970 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7jg6d\"/\"default-dockercfg-76422\"" Sep 18 20:06:16.987671 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.987622 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t"] Sep 18 20:06:16.998933 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.998897 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-sys\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:16.998933 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.998934 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-lib-modules\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:16.999203 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.998964 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n9cr\" (UniqueName: \"kubernetes.io/projected/db383831-5a6c-4690-8dd6-6eef521fcf5b-kube-api-access-6n9cr\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:16.999203 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.998984 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-proc\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:16.999203 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:16.999061 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-podres\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.100066 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.100019 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n9cr\" (UniqueName: \"kubernetes.io/projected/db383831-5a6c-4690-8dd6-6eef521fcf5b-kube-api-access-6n9cr\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.100066 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.100071 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-proc\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.100328 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.100135 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-podres\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.100328 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.100187 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-sys\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.100328 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.100214 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-lib-modules\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.100328 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.100217 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-proc\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.100328 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.100255 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-sys\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.100328 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.100278 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-podres\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.100522 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.100324 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db383831-5a6c-4690-8dd6-6eef521fcf5b-lib-modules\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.123333 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.123301 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n9cr\" (UniqueName: \"kubernetes.io/projected/db383831-5a6c-4690-8dd6-6eef521fcf5b-kube-api-access-6n9cr\") pod \"perf-node-gather-daemonset-pdb2t\" (UID: \"db383831-5a6c-4690-8dd6-6eef521fcf5b\") " pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.266221 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.266124 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:17.559801 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:17.559716 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t"] Sep 18 20:06:17.563932 ip-10-0-129-212 kubenswrapper[2554]: W0918 20:06:17.563886 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddb383831_5a6c_4690_8dd6_6eef521fcf5b.slice/crio-770a50302010bc61b340e455beca707db27123484d9868b3bd03e5891b278e6c WatchSource:0}: Error finding container 770a50302010bc61b340e455beca707db27123484d9868b3bd03e5891b278e6c: Status 404 returned error can't find the container with id 770a50302010bc61b340e455beca707db27123484d9868b3bd03e5891b278e6c Sep 18 20:06:18.448559 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:18.448516 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-ghj6f_c88b76e1-4c65-4f0c-aa2f-00cc0419a7e3/volume-data-source-validator/0.log" Sep 18 20:06:18.480828 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:18.480765 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" event={"ID":"db383831-5a6c-4690-8dd6-6eef521fcf5b","Type":"ContainerStarted","Data":"10a1e014e0b30b90e5a59f04a416789b77f0df772e088128d61fccd624e918ed"} Sep 18 20:06:18.481041 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:18.480851 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:18.481041 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:18.480869 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" event={"ID":"db383831-5a6c-4690-8dd6-6eef521fcf5b","Type":"ContainerStarted","Data":"770a50302010bc61b340e455beca707db27123484d9868b3bd03e5891b278e6c"} Sep 18 20:06:18.500935 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:18.500876 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" podStartSLOduration=2.500859257 podStartE2EDuration="2.500859257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:06:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:06:18.49893971 +0000 UTC m=+3124.403287488" watchObservedRunningTime="2026-09-18 20:06:18.500859257 +0000 UTC m=+3124.405207023" Sep 18 20:06:19.618346 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:19.618320 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rbmm8_9996e7d0-59d0-4457-8967-9a4c7eb67967/dns/0.log" Sep 18 20:06:19.645668 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:19.645625 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rbmm8_9996e7d0-59d0-4457-8967-9a4c7eb67967/kube-rbac-proxy/0.log" Sep 18 20:06:19.806565 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:19.806537 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v99gh_f6f93841-9b28-41ca-aab2-4caefd768842/dns-node-resolver/0.log" Sep 18 20:06:20.311291 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:20.311259 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9fpjr_bc2c0c9b-3578-46cf-a91e-b3eeef557f64/node-ca/0.log" Sep 18 20:06:21.114365 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:21.114331 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54788fddc8-7b577_2cee06b4-79a2-47d4-9443-e1d1353ccc82/router/0.log" Sep 18 20:06:21.458757 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:21.458729 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n2xhf_879e230b-7166-438e-b7d4-e88e9116a54b/serve-healthcheck-canary/0.log" Sep 18 20:06:22.115944 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:22.115913 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wqxts_15d1835a-24a8-4c8b-9c73-4afba0b63366/kube-rbac-proxy/0.log" Sep 18 20:06:22.136107 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:22.136067 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wqxts_15d1835a-24a8-4c8b-9c73-4afba0b63366/exporter/0.log" Sep 18 20:06:22.157107 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:22.157076 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wqxts_15d1835a-24a8-4c8b-9c73-4afba0b63366/extractor/0.log" Sep 18 20:06:24.494735 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:24.494705 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7jg6d/perf-node-gather-daemonset-pdb2t" Sep 18 20:06:24.693393 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:24.693359 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-8flxs_cbeb0e0e-5bc3-4b98-980d-56f7f5f03958/s3-init/0.log" Sep 18 20:06:24.712915 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:24.712886 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-8flxs_cbeb0e0e-5bc3-4b98-980d-56f7f5f03958/download-hf-model/0.log" Sep 18 20:06:31.380135 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:31.380108 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrv5j_059c4ca0-dcb8-4d39-b3b5-0dfad7944b18/kube-multus-additional-cni-plugins/0.log" Sep 18 20:06:31.413740 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:31.413708 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrv5j_059c4ca0-dcb8-4d39-b3b5-0dfad7944b18/egress-router-binary-copy/0.log" Sep 18 20:06:31.491546 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:31.491515 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrv5j_059c4ca0-dcb8-4d39-b3b5-0dfad7944b18/cni-plugins/0.log" Sep 18 20:06:31.562795 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:31.562754 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrv5j_059c4ca0-dcb8-4d39-b3b5-0dfad7944b18/bond-cni-plugin/0.log" Sep 18 20:06:31.584123 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:31.584066 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrv5j_059c4ca0-dcb8-4d39-b3b5-0dfad7944b18/routeoverride-cni/0.log" Sep 18 20:06:31.607784 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:31.607747 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrv5j_059c4ca0-dcb8-4d39-b3b5-0dfad7944b18/whereabouts-cni-bincopy/0.log" Sep 18 20:06:31.636302 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:31.636277 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrv5j_059c4ca0-dcb8-4d39-b3b5-0dfad7944b18/whereabouts-cni/0.log" Sep 18 20:06:31.708220 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:31.708185 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-n5qn8_d0ae961b-b362-4061-8067-faf028e26649/kube-multus/0.log" Sep 18 20:06:31.763265 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:31.763234 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dvkb2_0899c67e-8d96-4fa0-b310-b6641c91127b/network-metrics-daemon/0.log" Sep 18 20:06:31.783091 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:31.783066 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dvkb2_0899c67e-8d96-4fa0-b310-b6641c91127b/kube-rbac-proxy/0.log" Sep 18 20:06:32.872715 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:32.872662 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-controller/0.log" Sep 18 20:06:32.911460 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:32.911381 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/1.log" Sep 18 20:06:32.911460 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:32.911391 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovn-acl-logging/0.log" Sep 18 20:06:32.940468 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:32.940439 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/kube-rbac-proxy-node/0.log" Sep 18 20:06:32.966791 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:32.966756 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/kube-rbac-proxy-ovn-metrics/0.log" Sep 18 20:06:32.989280 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:32.989252 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/northd/0.log" Sep 18 20:06:33.017693 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:33.017666 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/nbdb/0.log" Sep 18 20:06:33.048041 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:33.048016 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/sbdb/0.log" Sep 18 20:06:33.109697 ip-10-0-129-212 kubenswrapper[2554]: I0918 20:06:33.109663 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqqtf_6e2f8a53-2ebd-4d26-9731-0d77648dfcf4/ovnkube-controller/0.log"